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p < 5 , and the sample with y. ., ^ 

dependent variate is divided in the -j . . itoapart 

with q degrees of freedom corresponding to the independent variate and the 
remaining part with n — q degrees of freedom. If a,-,-, bn denote the sums of 
squares and products corresponding to this division, then it is known that the 
joint distribution of a,-,* and hn , if the dependent vector variate is normal and 
actually, in the statistical sense, independent of the second vector variate, is 

I j I IQ I f- i i (a,.- + 6,.)1 da db 

(1) - — -L_£izi!-J- 

t-0 

where | A \ denotes the determinant of the matrix A = {a,;), and da the product 
of differentials da,j, and where for convenience the variance matrix of the 
dependent variate is taken to be the unit matrix. 

We make the transformation specified by 

A = WDW\ 

(2) 

A + B = WW\ 

where Z) is a diagonal matrix of the quantities r\ in descending order of magnitude, 
and W = [Wij] is a matrix (with transpose W') uniquely determined by (2) 
except for an ambiguity of sign for each column; this ambiguity can be eliminated 
by choosing positive elements in the first row. The Jacobian A of the trans¬ 
formation may be shown to be 

(3) = 2” 1 WW I n - r]). 

t-»l 7—<+l 

By direct substitution, we obtain from (1) the distribution 

= piwijyr^ = p(Wij)p{r]), 

where p(x) is a general notation^ for a distribution function in one or more 
vaiiates a:, (including the differential elements); for p{wtj) and p{r]) we have 

(4) p(wy) = Cl I WW' 1 exp [ - 1 £ dw, 

(5) p(r*i) = C^n (1 - II (A - dr^, 

i-i L y-»+i 

^ The probability symbol is not of course to be confused with the number p of components 
in the dependent variate. It should also be noted that for convenience p{Xi) is used to 
denote the joint probability for a set of quantities Xi , whereas p(xi) or p{x2) denotes the 
probability for the specified variate Ji or x^ considered separately. 





the constants C\ and C 2 being arranged to give unity on integration of p(M?»y) 
or p(r^i), i.e. we have 

(6) C, = 2 '^*"” »-♦>”fl (r[i(p - t)]/r[Kn - t)]l, 

(the Wij varying from — 00 to 00 except that wu > 0), and 

(7) C* = {r[Kn - i)]/(r[§(p - i)]r[i(9 - OlrlKn - g - t)])l. 

t-0 

3. Formal determination of the general distribution. The method to be 
adopted of obtaining the general distribution from the particular case quoted in 
equation (5) above is the same in principle as the one adopted by Fisher [7] in 
his derivation of the general distribution of the multiple correlation coefficient. 
Since the argument is more involved in the present problem, it will be presented 
first in formal probability terms, before the details of the solution are examined. 

We consider a transformation of the components of each vector variate to the 
true canonical components. I^et the observed ordinary correlation coefficients 
of these mutually independent components for one vector variate with the 
corresponding components of the second vector variate be denoted by s,-. The 
true correlations are the true canonical correlations p,. Then we have for the 
general canonical correlation distribution denoted by^ p(?\ | p,), the expression 

Piri 1 Pi) = / p(r<, Si I p<) 

J$i 

= / P(ri I Si , Pi)p(si I Pi) 

Jti 

= / p{ri 1 Si)p{si 1 pi)p(52 1 P 2 ) * • • p{sp I pp), 

the substitution p(ri | S;) for p{ri [ Si , pi) following from the sufficiency of the 
independent correlations Si of the corresponding pairs of canonical components, 
as statistics for the p,. We now define the function g{si , pi) by the relation 

p{si I pi) = p{si I Pi = 0) g{si , pi), 
whence we have the general solution 

Piri I p,) = / p(ri I Si)p{si I Pi = 0)g{si , pi)p(s 2 | P 2 = 0 )g(s 2 , P 2 ) • • • 

(8) = / p{ri , Si I Pi = 0)(7(«i , pi)g(s 2 , P 2 ) • * • 

J$i 

= piri I Pi = 0) / p{si 1 r» , Pi = 0)g{si , pi)g(s 2 , P 2 ) • • • 

Jai 

for p{ri I Pi) in terms of the special case p(ri | pi = 0). 


** Quantities to the right of the vertical stroke in a jirohahility bracket are given quanti¬ 
ties on which the prol)ability distribution depends. 
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Now according as the independent vector variate is considered as (a) a normal 
variate vith which the dependent variate is correlated, (b) a fixed vector in 
sample space (this includes the non-central means case) Fisher [7] has shown that 
the distribution of the multiple correlation R oi vl single dependent variate with 
an independent variate comprising m components is p{R | p = 0)g{Ry p), where 

^ ^ (a) p) = F(i n, J a; i m; p’‘R‘) (1 - p')*", 

(9) 

(b) g{R, p) = FH 


where we replace p’ by a parameter )3' in case (b), and the notation for hyper- 
geometric functions used is: 


F{a-,0;x) = 1 + ^ + 

p 


a{a -|- l)x^ 

m + 1 ) 2 ! 


+ 


F{oti, at; I3;x) = I + 


ai X 


«i(a;2 4- l)a2(a* + 1)*’ , 

;8(/3-fl)2! 


It follows that we may write < 7 ( 51 , pi) above in the form 

(^) !7(®i > Pi) ~ 2 ! (I ~ Pi)^ > 

(b) <7(si,Pi) = 

by putting m = 1 in (9), (the signs of the are arbitrary, so that we are essen¬ 
tially concerned, as in the multiple correlation distribution, with the squares 
of the correlations). From these series expansions the integral in ( 8 ) consists of 
terms corresponding to the conditional moments, for any set of positive integers 

1] j t2 ) ' ' * j Ip i 

, b , • • • , g - • • • (4)*' 1 ril 

Jai 

In the pai’ticular case when only pi 9 ^ 0, the moments p{t) = E{{siy \ r<) from 
the single factor g{si , pi) are all that ririse, but in the general case it is important 
to notice that the quantities .s\ , while statistically independent when unrestricted, 
are no longer independent for the conditional distribution p(.Si | r<, p,- = 0). 
This completes the formal solution. It remains to evaluate ii{ti , ^ , • • • , ^p). 


4. The conditional moment /x(/i, , • • • , ^p). First of all we note from the 

choice of the components of the dependent vector variate, applying the analysis 
of section 2 to such components, that the multiple correlation Ri between the 
ith component and the q components of the independent variate is given by 

R\ = o<i7 {da + &•») = (AiTi + a%rl -f- • • • -F , 


Otij — Wif/ V ^ ^ ... ^ w]p). 


where 
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To obtain the distribution of the from that of the , we note that the Wi, 
distribution (4) is normal (allowing for convenience Wu to vary from — «o to oo) 
except for the “linkage factor’’ 

2 -b(-p) ( 

<••0 

Hence if we transform to the variables cu , 6^ defined by 

Cii = w]i + w^i2 + • * * + vUp , 


( 11 ) 


an = cos Oil , 

a ,-2 = sin da cos 0 , 2 , 

aiz = sin On sin 0,2 cos 0 « , 


aip = sin 0,1 sin 0,2 sin 0,3 • • • sin 0 ,,p_i, 
the sets cu , Bxj which for normal wa would all be independent with distributions: 

p{cii) — X distribution with p degrees of freedom, 

(12) p(0i;) sin^“‘^^0,;-d0,/, 

(0 < dxj < TT fory = 1, 2, • • • p — 2; 0 < 0,-,p_i < 2 tt), 


in general retain their independence for given i, but the linkage factor results 
in an elevation of the distributions to n degrees of freedom, and a linkage 
factor for the 0 ,-, distributions of 


(13) 

where 


I A 


¥t /r[^(p - 0]r[i«]\ 
- OITOp]/’ 


A = {a<iayi + «*2ai2 + • • • + 

We may now, having obtained the distribution of the a,;, note their geometri¬ 
cal interpretation. Let us denote the p components of the dependent variate 
in n-dimensional sample space by the p vectors 5i, ^ 2 , * * * , • Let the p 

orthogonal canonical components corresponding to the sample canonical correla¬ 
tions Ti be denoted by the p unit vectors Xi, JC 2 , • • •, Xp . I^et the corresponding 
components for the independent variate be n,-, y,. The “linkage factor” 
merely represents the allowance that must be made in the mutual relations of the 
^-vectors for the fact that while they must lie in the p-space of the x-vectors, 
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they really belong to the original n-space. We may identify the w,/ with the 
coefficients in the equation 

(14) = w,iXi + w^2X2 + • • • + WtpXp , 
where 

+ w\2 + * * * + w\p 

is a X* with n, and not p, degrees of freedom. If we now suppose for convenience 
to be a unit vector, wc have in place of (14) 

(15) = oftiXi + ai2X2 + * • * + oitpXp , 

with a projection, on the g-space of the y-vectors, of C», say, where 

{* = OtiTiyi + ax2T^2 -j- ... ^ ctxpTpyp , 

and hence, as already noted in the algebraic derivation, 

R\ = (X. • QVC. = + a A + .. . + a\pT\ , 

where (^ • 0 denotes a scalar product. The linkage factor (13) indicates that 
the \x vectors in (15) are not independent in the p-space of the x-vectors, the 
distribution of their mutual configuration being dete^’mined by n-space. 

This interpretation enables us to determine the moments of the distribution 
p(5,1 r»). For if corresponding to (15) we write 

(16) lit = ^xlVl + -f . . . PxqVq , 

then 

(17) S» = Oixlfixl^l "i" “I" ’ ' * "I” fXxp^xpTp • 

If we are consideiing case (a), the relations of the ii» to the y-vectors in g-space 
will be similar to the relations of the ^, to the x-vectors in p-space. In case (b), 
however, the n*, which represent the true canonical components of a set of q 
fixed vectors, must remain strictly orthogonal to each other although their 
relation to the y-vectors can vary. This means that the relations of the n, 
to the y-vectors are determined by a random rotation of a rigid orthogonal set 
of q vectors in case (b). We may note that if in case (a) we allowed n to tend to 
infinity, the n, would also become rigidly orthogonal, so that the solution in case 
(b) may conveniently be obtained from case (a) by retaining the same distribu¬ 
tion of the ax , and for the /8, letting n —^ . 

Thus in either case the moments of the 5, can be obtained from (17) in terms 
of the moments of a,, and jS,,, two independent sets of coefficients for which the 
distribution of each set is known. The above comments suffice theoretically to 
complete the required solution for (si)*^®*)** • • * (sp)*** is a function of a,, and 
0x} ; the axj and the corresponding linkage factor can be expressed in terms of 
sin Bxj and cos Bxj , and similarly for the jS*, in terms of, say, sin and cos 0,^, 
and integration oairicd out over the Bxj and (t)xj. This method is unfortunately 
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too cumbersome algebraically to be of any practical value except in the case of 
one non-zero root. This case is considered separately before the general case is 
discussed further. 


6. The case of only one non-zero root. Here we only require /u(0 and a 
comparatively simple solution is possible, the linkages uithin the and m sets 
being irrelevant. We have in fact, if tf/ is the angle between tii and (i, (where 
Cl was the projection of Ci in the ^-space), that ^ is a random angle in the g-space, 
since the Pa sets are independent. Hence in this particular case we may 

conveniently write si = Rl cos V? which is just the transformation used to obtain 
the distribution of the multiple correlation Rl . Thus we may replace (10) by 
(9), where Rl = ahrl + aur\ + . •. + aiprp, and 


(R?)' = E 

1 il + U 2 +* • 


Wi!u 2 !* • • 

• cos^“‘ On cos=“*e ,2 6,^ • 


where the expected value of the trigonometric term is evaluated as 


(18) 


'r(Mi + i)r(M 2 + 

r(|)r(j) ... j 


Tihp) 

r(-|p + 1) • 


We have now obtained the distribution, (pj = • • • = p„ = 0), 

p(ri 1 PI 0) = p{r, 1 p. = 0) - , itj, • • , 

where p(r,-1 p,- = 0) is given by (5); and in case (a) 

_ niP)T{k) fr r rfa,- + i) 1 

nip + 0r(k + t) f -1 L r(|)w,-! J ’ 


and in case (b) 


C(mi, Ut, ••■) 


= e 


kiAY 


r(|n + <)r(^p)r(J< 7 ) A [ r(M,- + i)] 

r(ja)r(ip + <)r(k + 0 r-i L r(i)M,! J’ 


where mi + ife + • • • + Mp is denoted by t. .uj , • • • denotes summation of 
all m’s from 0 to «>. The solution in either case contains a generalized hyper¬ 
geometric function. If we denote the general series 

■r(ax + <)r(a 2 + 1 ) r(n)r(r 2 ) A r r(/3,- + M,)x“n\ 

r(ai)r(a2) r(r,-l-<)r(r2-|-<)iiL r(A)«,! Jj’ 

/r(a + <) r(n)r(r2) Afm + MMni 

r(a) r(ri -I- orfrs-h 0 M L r(ft)M;! Jj 


Eui.t 


.{ 


F{oti , 0(2 J > ^2 > * * * > > ri, r2 > , Xo, • • • , ^p)> 

F{ct \ ^1 f ^2 1 * * * > ^p > n > ^2 J , 372 , * * * ) ^p) 


by 
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(19) 


respectively (see [ 8 , p. 300, example 22]), then we have in case (a) 
p(r.-1 Pi 0) = p(ri I Pi = 0)(1 — p?)*" 
y.P{\n, Jn; J, • • • hi', plrl, p\r\, ■ ■ • , Pir*,), 

and in case (b) 

p(r< I A 5^ 0) = vifi I Pi = 0)e~*^i 


( 20 ) 


XF(i »; iP, k; iArl 


An alternative operational form is obtained by noting that the sum of terms for 
given ^ = ui -h zi 2 -h * * * ■+■ Up is generated by means of the coefficient of in 

11(1 - p}rU)~*, 

y -1 

where for definiteness we consider case (a). Hence if we write 

r(ri)r(r 2 ) 


P(i, I.r,, 2 ,) Et r(a,)r(aj) r(ri + <)r(rr+1) 


a:‘, 


we have 


P(in, ^n; i,i,’'’A'f iP, iQi Pi^i , pirl, , p\r\) 

(21) p 

= eF{in, in; ip, iq; 0 "*) JI (1 “ plrU)-^, 

J -1 

where 0 denotes the operation of taking the term independent of z (this might 
possibly be done by multiplication by 2 ”^ and evaluation of a suitable contour 
integral, but in the use of this formula here the operation 0 has been carried out 
directly). 

It is of some interest to examine a simple case, and, incidentally, to check that 

f P(ri I Pi) = 1. 

Jri 

If we take p = 2, g = 3, we obtain for p(r?, rj | pi = pj = 0) the form 

i(n - 2)(n - 3)(n - 4)(1 - r?)‘’-’(l - r^)‘’-’(r? - rl) drl drl. 


Considering the distribution (19) with p = 2, ^ = 3, and taking the most ele¬ 
mentary case n = 6 , we obtain on integration of t\ from 0 to rj, 

P(n 1 pi) = 6(rJ)®dn(l - pj)’ Eui.vt 

r(i) m + «i)r(^ + u,)Ur\y 
r(|+0 m)r(i)ttii(«» + 2)i<i ’ 

where t = ui + ih • Now from the identity (1 — — a;)* = 1 — x, the 
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coefficient of x*'”, (< > 0 ), is zero in the expansion of the left-hand side. This 
provides the identity, for all t > 0 , 

^ r(i + Mi)r(j + t - Ui)i -1 _ ra -f < -t- 1 ) _ r(i -I- < -I- 2 ) 
r(i)«iim)(«,-i- 2 )! r(i)(t4-i)i r(i)(f + 2 )! 

^ W + 3)r(| + 1 ) 
mm+ 3) ’ 


r(j + Mi)r(i H- <-«,)_< + 3 r(| + 0 


r(i)«i!r(i)(M 2 + 2)1 


3 r(|)r(«-f-3)' 


Hence 


P(n I pi) = 6(rS)’drUl - plY E. ^ (plrlY 

(23) ^ r(3+ - t)^Mp + 3 ) 

= (1 - pj)’drjd/drn(r?)’(l - P^r?)"’!, 

which obviously gives unity on integration of t\ from 0 to 1. In purely algebraic 
form 

(24) p(n 1 pi) = 3(1 ~ dT\/{l ~ p\t\)\ 

Alternatively, making use of formulae (21), we have for the same case p = 2, 
5 = 3, n = 6, the distribution 

(25) 6 (n - rl) dr? dri(l - plY eF(3, 3; ?; 3"“) (1 - plrhy* (1 - plrhr*. 
Integrating with respect to rl from 0 to r? , we obtmn 


.2,^-J2ri , 4[(1 - p^iz)^ - 1]' 


(26) 6 drla - plyeF{3, 3; 3; 3 -') (1 - plrUT 


Discarding the term for which the irrational expression (1 — pirlz)* cancels, 
and hence leaves no terms independent of 2 , we obtain the distribution pin j pi) 
given in (23) or (24) by selection of the appropriate terms. We may further 
integrate directly the expression above with respect to rl , and after discarding 
again irrelevant terms we obtain 

(27) 6(1 - piyeF(3, 3; i; «"*) (1 - p?z)‘|, 

which is readily ascertained to be unity. 

6 . More than one non-zero root. In the general case the factor multiplying 
p(^< 1 Pi = 0 ) is rather remarkable in being symmetrical in both the set r* and 
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the set pi. As n increases, the convergence of ri to pi, r 2 to p 2 , etc. when the 
Pi are also arranged in descending order of magnitude must result from the 
restriction n > r 2 > • • • > . The limiting distribution has been discussed 

by Hsu [9]. 

In view of the algebraic difficulty of obtaining p(<i, , • • • , O by direct 

integration, an unsymmetric method of obtaining the moments was developed. 
This is fairly tractable in the case of two non-zero roots. The second set vhj 
of the original variables is transformed by an orthogonal transformation such 
that the first new variable of the second set is determined by the correlation 
between wn and w^j . We may write, for example, 


wn = (wiiWii + w^W22 + ■ ■ + • • • + JOip)*, 


. {= 


-Wii(wi2 H-h Wip)Wii . , 

-+ W12W22 -r 


wh 


(28) Wii -^ w tp)(wn H-+ 


2 

mi 


ms = 


-m2(t0i3 + • • • + Wlp)W22 




+ W 13 W 23 + 


f(Wi2 + • • • + 'U>ip)(Wi8 + • • • + W/lpjV 


W 12 


which conversely we can at once express as a relation of the W 2 /, in terms of the 
W 2 S, (since the reciprocal of an orthogonal matrix is simply its transpose). If 
we write 


«21 — mi/l(W2iy + (W22y -[-•••+ (ti;2p)Y> 
0:22 = w 22 /[{m-d^ + {^22)^ + • • * + ('*^ 2 p)^]S 


and write further 

ai = cos ^11,02 = cos 012 , • ’ • , 61 = cos dn , 

62 = cos 022 , • • • , where an = cos 02 i, = sin 02 i cos 022 , • * * 

we have in particular 

0121 = ai 6 i - &2 \/(l - a?) ^(1 - bj), 

(30) Ota = 0261 -\/(l - al) + “ 102 ^ \/(l — 61 ) 

- h V(1 - al) V(1 - b\) V(1 - bl), 

where the distribution of the o’s and b*s is proportional to 

{(1 - da.) {(1 - • • • 1(1 - b?)*'”"” d 6 ,} {(1 - 6 |)‘<’^‘> d62l • ■ • . 
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For the reasons discussed in section 4, it will be noticed that only the distribu¬ 
tion of hi in the a, h set is affected by the linkage factor. By such methods the 
expressions 

nil, 1) ^ I n], m(2 , 1) 3 S{s}8| 1 r,} 

were fairly readily obtained. If we introduce the notation 

^ib — Z) Ski s etc., 

*•-1 

and also symbols for the products of the a and moments, viz. 

3 E[c^10^2} ^{^ 11 ^ 12 ), ^ 2 ) ~ 

etc., we may list the moments /xC^i , ^ , * * * , ^p) as in Appendix I, which gives all 
moments up to the fourth order in terms of the a and ^ moments (the numerical 
coefficients arise from the numbers of ways of forming the two-way partitions). 
‘‘Half-factors’* corresponding to the a moments are listed in Appendix II against 
their appropriate symbol, the corresponding factors coming from the jS moments 
being obtained in case (a) by writing q for p and in case (b) by writing also*^ 
n —> 00 . Thus in case (a) 

f n + 2 1 r n + 2 1 „ 

Lnp(p + 2)J [_nq{q + 2)J 

, /r np + n — 2 1 r ng + n - 2 1 

\Up(p + 2)(p - 1)J Lng(g + 2)(g - 1)J 
. 2 r ^ P) 1 2 ^ 

Up(p + 2)(p - 1) J \nq{q + 2)(g - 1) Jj ' 

and in case (b) 

[^ip(^2)]L((7 + 2)]'^“ 

(33) 

, / np + n - 2)(g + 1) + 2(n - p) \ 

\np(p + 2)(p - l)g(g + 2)iq - 1)/ “ ‘ 

By means of the transformation (28) it is possible to develop the moments 
p{ti , fe) in the case of two non-zero roots, though in obtaining the results quoted 
in Appendix II, where the formula for /i(3, 1) and /i(2, 2) are included, it was 
found convenient to supplement this method with the devices mentioned in the 

• It should be remembered that we have assumed p < q. If p > g, we interchange 
the dependent and independent vector variates, and hence must interchange p and q in 
these moment formulae, p(<g) now corresponding to the independent variate. 


m(i. 1) = 

(32) 
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next section. In the case of more than two non-zero roots, it is theoretically 
possible to carry out a further transformation on the Wzi variates, but with the 
“partial” variates W\j .2 ^ v>ij — huW^j, where 

hi 2 = {wnW 2 i 4- Wi 2 {i 022 + • • •)/{v)h + WI 2 + • • •), 

as coefficients. This enables us to express wsj in terms of new variables, of 
which the first is related to the partial correlation of w$j with wij for given W 2 j , 
i.e. to the second correlation factor which depends on the ‘linkage”; and so on. 
This method is, however, again too cumbersome to be of much use, and a more 
rapid method of evaluating fi(ti , < 2 , * • * , ^p) in general is desirable. This problem 
has not been entirely solved to the author’s satisfaction in this paper, although 
in the concluding section aie mentioned devices which have been found useful, 
and which enabled the terms for the remaining third-order moment 1> 1) 
to be completed and added to Appendix II. 

7. Relations among the a-moments. Equation (15) defining the a’s, the 
being random vectors in the p-space of the x-vectors except for their mutual 
configuration being determined by the properties of n-space, may be used to 
provide relations among the a-moments. Thus in addition to the identities 

(34) all + ai2 + • • • + a'lp =1, (t = 1, 2, • • • , p), 

the correlation of any with a fixed vector in the p-space, e.g. with Xi or with 

(Xi + X 2 )/\/ 2 , is a random correlation in p-space, wliereas the correlation of any 

lx with any other is a random correlation in n-space. The use of these facts 

is best illustrated by an example and equations sufficient to determine the 

six a-moments required for /x(l, 1, 1) will be derived. 

For convenience, denote the required mean values of 

222222222 2 2 
Ot\lOL2l(^n y «11«21«32 , «11«22«33 , Cltl\Otl2Ct2lOi'2ZOLZl , (XnOCl20t2lOi22asZ , «liai2a22«23«31«33 

by A, C, Z), Ej F respectively. Multiply the second-order quantities ahah , 
ahalz, cL\iai 2 a 2 ia 22 by expression (34) for t = 3; since tliis expression is identically 
unity, the consequent mean values are unaltered. This gives the three relations 

A + {p — l)B = (w + 2)/{np(p + 2)!, 

. 4 + 3(p - l)B + (p - l)(p - 2)C = 1/p, 

^ + (p - 1)B + 2 (p - l)(p - 2)D 

+ (p - l)(p - 2)(p - m = l/(np). 
The moment A is the mean of the triple product of the squared scalar products 
of and with Xi. The same value must be realized with any other fixed 
vector in the p-space, e.g. with either (x, + x^/y/2 or with (Xi -b Xj d- • • • -f Xp) 
/\/p. This gives two relations 

A - B - 4.D = Q 

(p + DA - 3B - 120 - (p - 2) (C -f QE + SF) = 0. 


( 36 ) 
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A final linearly independent relation is obtained from the mean triple product of 
(?i • ^ 2 ), (5i * &), (?2 * ts), which depends solely on the internal configuration of 
, h and , and is easily shown (e.g. choose to coincide with one of the original 
axes of the n-space) to be 1/n^. This gives 

(37) pA + 3p(p - 1)D + p(p -Dip- 2)F = l/n\ 

The equations contained in (35), (36) and (37) determine Ay B, C, D, E andF. 

Similar equations could evidently be constructed for the higher-order moments, 
e.g. for the terms required for /li( 2, 1, 1) or /i(l, 1, 1,1), but the numbers of such 
terms increase rapidly. From Appendix I it will be seen that there are 24 
distinct terms in /i(2,1,1) and 16 in /x(l, 1,1,1). 


Appendix I. 


«», 1 ) - s. Q + 2 S ..|0 2) + 2(1 J)} 

«(2, 1) = «.(2) + Sn{(‘ 2)+ “(2 0} 

+ " 2 ) + ‘K 1 1 ')} 

p( 3 ,l) +So 2)+ ‘*(2 1)} 

+ 2 S.{l 6 (* ^)+ 2 O 0 0 ) 

+ 2S„,{i5(‘ 2 ,)+45(^ 2 I V“(I 1 

+ 24S.,u{i5 0 ^ ^ 2 ) + '" (I I ^ 0} 

0(2,2)-S.(*) + S»{l2(^ 2)+“(3 !)} 

+ 2So{(‘ 4 )+ ‘*(2 0 + 

+ 2 2) + “(2 ^ 2)+”(l i 0 + *’(l I 

+ 24Smi{90 ^ 2 2 ) + ” (l 1 ^ 2 ) + “ (I I I 
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m(1, 1, 1) = 2 + 1 3 2 • + 12 1 1 


+ 6aSiii 


2 . • 


+61 1 . +8 • 1 1 


m(2, 1, 1) — >84 ( 2 1 + 8 ^ai *12(2 •1 + 6(2 


+ 16 1 1+41 1 U + 2&2 ^ . 2+62 



+ 16 1 1 + 12 1 1 U + 2 & 1 J • 2 • + 12 2 




1 1 2 


+ 16 1 1 . + 2 • 1 1 + 24 1 1 


2 1 1 


+ 48 ^1 1 j + 48 ^1 1 • j + 24 1 ij + 32 1 1 

r /2 2 . /2 2 • .\ /I 1 1 1\ 


2 1 1 


+ 24/Sjjii s 3 


1 1 + 24 1 1 


/2 1 1 A /I 1 . 2' 

+ 24(- 1 1 .j + 48(- 1 1 • 


'2i f r2 .1 

m(1, 1, 1, 1) = S 4 + 2 ! +24 


2 • 1 1 


{ (2 


+ 2S^W3 " 0 +24 ; t + 8 


1 1 ' ^ 1 1 

11 11 

/ \ J / 


+ 2/8211 *s 6 


1 • 1 

1 1 • 

1 1 • 

1 • 1 
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fi 1 .] 


'2 • 


+ 96 

• 1 1 

1 • 1 

+ 12 

• 1 1 

• 1 1 



[2 • -J 


2 . . 



+ 24Sim S 


+ 12 


1 1 
1 1 


1 1 


+ 32 


1 1 • 

• 1 1 

1 • 1 


+ 12 


1 iJ 


1 - • 

• 2 j 



'2 • • 



fl 1 

•1 


• 1 1 • 

• 1 1 • 

+ 48 

• 1 

• 


[■ . . 2 j 



.1 • 

• ij 

J 


Appendix II. 

/2\ n + 2 /2 A np + w — 2 /l l\ —(ra — p) 

\2/ Mp(p + 2) ’ \- 2y np(p + 2)(p - 1) ’ V 1/ np(P + 2)(p — 1) ’ 


A\_ 3(w + 4) _ /4 A 3(np + 3n - 4) 

\2/np(n + 2)(p + 4) ’ \- 2/np(p + 2)(p + 4)(p — 1) ’ 

/2 2\ np + n + 2p — 4 /2 2 A np + 3 ti — 4 

\2 ./mp(p + 2)(p + 4)(p- 1)’V • 2/ np(p + 2)(p + 4)(p - 1) ’ 

-3(w-p) (\ 1 2\ -(n-p) 

\1 1/ np(p + 2)(p + 4)(p - 1) ’ \1 1 • / np(p + 2)(p + 4)(p - 1) ’ 

/6\ 15(w + 6) /6 A 15(7ip + 5w - 6) 

\2/np(p + 2)(p + 4)(p + 6) ’ 2/»p(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/4 2\ 3(np + n + 4p — 6) 

\2 •/ np(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/4 2\_ 3(wp + 3w + 2p — 6) 

\' 2/ np(p + 2)(p + 4)(p + 6)(p - 1) • 

/4 2 A_ 3(np + 5n — 6) _ 

V • V np(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/2 2 2\ wp + 3n + 2p - 6 

\2 ■ ■) npip + 2)(p + 4)(p + 6)(p - 1) ’ 

/2 2 2 A_ np + 5n — 6 _ 

• 2/ np(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/6 l\_ -»I5(n - p) _ 

Vl 1/ np(p + 2)p + 4)(p + 6)(p - 1) ’ 
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/3 3\ _ —9(n — p) _ 

\1 1/ np(p + 2)(p + 4)(p + 6)(p — 1) ’ 

/3 1 2\_ — 3(n — p) _ 

\1 1 •/ wp(p + 2)(p + 4)(p + 6)(p — 1) ’ 

/l 1 4\ _ -3(w - p ) _ 

\1 ] •/ »p(p + 2)(p + 4)(p + 6)(p - 1) ’ 

A 1 2 2\ -(n - p) 

V 1 • • / np(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/ 4\_ 9(w + 4)(w + 6)_ 

\4/ n(n + 2)p(p + 2)(p + 4)(p + 6) ’ 

/4 •V oln^p + 3)(p + 5) +2n(p + l)(p + 3) - 8(2p + 3)} 

\- 4/j n(n + 2)p(p + 2)(p + 4)(p + 6)(p - l)(p + 1) 

/2 2\ 3{w^(p + 3) + 6w(p + 1) ~f~ 8(p — 3)} 

V4 ■/ n(n + 2)p(p + 2)(p + 4)(p + 6)(p - 1) ’ 

(2 2\ n®(p" + 4p + 15) + 6n(p + l)(p - 3) + 4(5p’' + 2p - 6) 

V2 2 ) n{n + 2)p(p + 2)(p + 4)(p + 6)(p - l)(p + 1) 

/4 • A 3{w\p + 3)(p + 5) + 2n{p + l)(p + 3 ) — 8(2p + 3)} 

\- 2 2/ n(n + 2)p(p + 2)(p + 4)(p + 6)(p — l)(p + 1) 

(2 2 A n\p + 3)^ + 2n(p + l)(2p + 3) + 4(p'‘ - 4p - 6) 

V2 • 2/ n(n + 2)p(p + 2)(p + 4)(p + 6)(p - l)(p + 1) 

(2 2 • A n\p + 3)( p + 5) + 2n(p + l)(p + 3) - 8(2p + 3) 

\- -2 2/ n{n + 2)p(p + 2)(p + 4)(p + 6)(p - l)(p + 1) ’ 

/3 l\ —9(n — p)(» + 4) 

\3 ljn(n+ 2)p(p + 2)(p + 4)(p + 6)(p - 1) ’ 

/3 A —9(n — p)inp + 3n + 2p )__ 

\1 3/ nin + 2)p(p + 2)(p + 4)(p + 6)(p - l)(p + 1) ’ 

A 1 2'\_— ( n — p)(np — 3w + 8p + 12) _ 

\1 1 2/ «(« + 2)p(p + 2j{p + 4)(p + 6)(p - l)(p + 1) ’ 

/3 1 A_ —3(w — p)(np + 3n + 2p) 

\1 1 2 ) n{n + 2)p(p - 2)(p + 4)(p + 6)(p - l)(p + f)’ 

A 12 A__ — (w — p)( n p + 3n + 2p) _ 

\1 1 • 2/ n(n + 2)p(p“+ 2)(p~+ 4)(p + 6)(p - l)(p + 1) ’ 

A 1 1 l\_ 3(ra - p)(n - p - 2) _ 

\1 1 1 n(n + 2)p(p + 2)(p + 4)(p 4-6)(p — l)(p 4-lA 
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(n + 2)(n + 4) 
U/ + 2)(p + 4) 


■ 63 - 


(n + 2)(np + 3n — 4) 


,*p(p + 2)(p + 4)(p - 1) ’ 


C'). 


(p» 4- 3p - 2) - 6ra(p + 2) + 16 

•^(P + 2)(p + 4)(p - l)(p - 2) ’ 
-(n — p)(n + 2) 


.= p(p + 2)(p + 4)(p - 1)’ 


(! ' ;)» 


(■' 3 .-. 


— (n — p)(np + 2ra — 4) 


n^pCp + 2)(p + 4)(p - l)(p - 2), 
(n - p)(2n - p) 


.'•‘p(p + 2)(p + 4)(p - l)(p - 2)' 
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ON THE THEORY OF MARKOFF CHAINS 


By Elliott W. Montboll 
University of Pittsburgh 

1. Summary. Although there exists voluminous literature on the theory of 
probability of independent events, and powerful techniques have been developed 
for the analysis of most of the interesting problems in this field, the theory of 
probability of dependent events has been father neglected. The first detailed 
investigations in this subject were published by A. Markoff [1]. S. Bernstein [2] 
has extended the fundamental limit theorems to chains of dependent events. 
The most extensive exposition of this field has been made by M. Fr^chet [3]. 

In the present paper we shall develop methods of averaging functions over 
chains of dependent variables and find the probability distribution of these 
functions. It will be shown that for certain types of chains these averages and 
distribution functions can be expressed in terms of the characteristic values and 
vectors of a certain operator equation. Many of the methods discussed here 
have been applied to problems in statistical mechanics [4,5, 6,7,8]. The most 
important application has been made by 1^. Onsagcr [8j who proved rigorously 
(on the basis of a simplified model) that Boltzmann^s energy distribution in a 
solid with cooperative elements leads to a phase transition. The first explicit 
application of linear operator theory (through matrices and integral equations) to 
probability chains has apparently been made by Hostinsky [9]. 

2. Introductory Renuu:ks. Suppose there exists a chain of events each of 
which might lead to one of v possible results, and which arc correlated in such a 
manner that the probability of n successive events leading to a chain of results 


ai, a2 , • • * , a„ 

is proportional to 

Pnicti , a2 , • • • , a„). 

The probability of a given function F{ai , ^ 2 , • • • , «»») having a value correspond¬ 
ing to the sequence of a^a would be proportional to 

F{ai , a2 , • • • , On) Pn{(xi ,*••,«») 

and its average value over all configurations of the chain would be 

(1) P = Fi/Fq ^ F{ai,a2 f • •• f a,)P»(ai, a*, • • • , an)/ 2 Pn(ai , • • • , a,) 

laiJ 


Fm «■ 2 [F(ai , On)]*" Pn(otl , • • * , an) 

(«/) 


where 

(la) 
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and the summation extends over all values of 

{ay} = (ai,a2, ••• ,an). 

The probability of a result ai of the first event leading to a result a» of the 
nth event is 

(2) > «n) = (l/Fo) X) Fnioti , a2, • • * , aj. 

' ' «*,***.On-l 

In order to find the probability of a given function F{ai , • • • , an) having a 
value between { and { + /i it is useful to know the moments and Thiele semi¬ 
invariants of F(ai, • • • , an). Both of these functions of F can be calculated 
from 

/ 3 N Zn(x) = X) Pn(oti , • • • , an) CXp {xF(ai , a 2 , • • • , an)) . 

^ ' («,1 

Obviously 

(4) Fn, = Urn Zn{x)ldx”'. 

It is known [10] that the mth Thiele semi-invariant is given by 

(5) Am = lim d”" log Zn{x)ldx”', 

' ' x—O 

In the notation of Cramer Zn{i<j^)/Zn{0) = /(w), the characteristic function of F. 

If G{z) is defined so that -f- /i) — (j({) is the probability that the function 
F(ai, • • • , an) has a value between J < F(ai, • • • , an) < ^ + h, then it is well 
known that [5] if Giz) is continuous at x = { and x = ^ + h 

(6) G{k + h) — Gft) = ^ lim / -- exp [log/(<tf)] dw 

where 

(6a) log/(«) = i: = t, A„(ico)7m! + oiJ). 

m—1 Ml m—1 

When the derivative of G({) with respect to { exists, the probability of 


F(,ai, ,a„) 

having a value between J and f is 

(6b) d£ = (dO/di) df = ^ lim fexp (f) A„(t«)7TOl|e"'"* Ao. 

r-*oo J— T ^m«i J 

From (4) 


CO 



A»(to)7ffi! = -log ^„(0) + lim e-*"®'®* 

x-»0 


( 7 ) 


log Zn(x). 
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Since, for a constant c independent of x, 

= f(x + c) 

we have 


( 8 ) 


E A„(to)7wH = log {Z»(w)/Z,(0)l, 

maal 


and from (6) 

(9) Gte + h) 


G(f) = o4;- / 

r-*oo J— 


^ - e^^^)Zn(io)) do) 


a)Zn(O) 


Equations (3), (4), (5) and (9) indicate that much information concerning a 
chain of correlated events can be obtained from a knowledge of Znix), We shall 
now introduce procedures for the determination of Z„(x) for several general forms 
of P(ai , • • • , an). 

When «is a continuous variable, the results of this section and those to follow 
are easily generalized by replacing the summations operations over all values 
of the a’s by integrals, and by replacing the matrix equations of the next section 
by integral equations. 


3, Simple Chains, Pn(ai, •••,«„)= JI P(«;. 

/-I 

a. General theory. By a simple chain we shall mean a sequence of events, 
each of which leads to one of v possible results and which occur in such a manner 
that if the result of the fcth event is a^ , the probability of the {k + l)st one 
3 delding a result a^+i to proportional to , a^+i). This implies that the 
probability of the occurrence of the sequence of results 

ai , a2 , • • * , an 

is 

n—1 / n—1 

(10) np( cii , a(,+i) /i: n p(a., a<+i), 

t—1 / I a,} I 

and the probability of a first result a \, leading to an nth result an is 

(11) Pn(ai,a„)= 2 np(“>>“j+i)/2 np(<*»i«>+i)- 

The summations are to be extended over all v possible values of each a* indicated 
on the summation indices. Chains of this type are sometimes called simple 
Markoff chains after the first author who studied them systematically. 

From (1), the average value of a function F(ai, • • • , an) is 

n—1 

23 • • • z) ^(“1 .•••.«») n p(«i. “)+i) 

^ ->=!!- . 

23 ••• 23 nP(a». ay+i) 

«1 «n y-1 


(12) 
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Many chain functions F{ai , • • • , otn) of interest are either additive or multiplica¬ 
tive and of one of the forms 

(13a) a) Fi{ai , • • • , «„) = h{ai , h{a 2 , as) + • • * + , an) 

(13b) b) Ft^ai , • • • , a„) = g{ai , ^ 2 ) g{ct2 , as) • • • gf(afn-i, an). 

In case (b) it is convenient to define a new function h{ai , a,) by 

(14) g{a ^, ay) = exp[xh(a^ , a,)] 

and in both cases to consider a function of the form 

n-l 

as) z„(x) = E n p{a,, Uj+i) exp [xhiaj, a,•+»)], 

(«.l t-i 

for then the values of Fi and F 2 averaged over the entire chain arc given by 
(16a) <Fi>av. — lim d log Zn{x)ldx 

x-*0 

and 


(16b) 


<F2>av. - Zn(l)/Zn(0). 


When n is large, the direct evaluation of (15) may become quite dilficult 
because of the large number of variables involved. As an alternative we shall 
now introduce a procedure that is based on the observation that Zn(x) is the 
sum of the elements of the nth power of the matrix 


(17) 


p*(i, 1) Px(i,2) ••• pz{i,vy 

p,(2,l) 27,(2,2) ... p,(2, ^) 


[vxil'y 1) Px{Vy 2) 


Px{Vy v)j 


where the elements Px(a, 0) are defined as 


(18) Px(a, 0) = p(a, 0) exp[xhiay /?)]. 

a and 0 range over the same set of values as one of the “result’' parameters 
ay ; and each of the v possible results is represented by a unique integer of the set 
1,2, ... , */. Thus Zn(x) = sum of elements of To employ this observa¬ 

tion to advantage, let us consider the characteristic values and vectors of the 
matrix P, . It is well known that if the characteristic* values are simple the 
characteristic vectors form a biorthogonal set; that is, if 

(19a) = {<Pi,xWy ^i>..x(2), ... , (f - 1, 2, . •. , I/), 


and 

(19b) 


= 


^t.*(l) 

^i.x(2) 

J'x.x{v)_ 
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satisfy the operator equations 
(20a) 

(20b) Px • ^..x = X,.x^,.x 

where X,, is the ith characteristic value of (17), then 

r 

= 2 »>..x(a)f ,.*(«) = 0 when i 7^ j. 

a—1 

We shall for convenience always assume that the ^’s and ^^s are normalized: 


so that in general: 

(21) ^»x*^;.x = = 


^,.x • = 1 

0 when i 9 ^ j 


1 when ^ = j. 

It is well known from matrix theory that one can expand a matrix element as 


Px(a, /3) = Z) Kx<Pt,x(0)'Pt.xM 


( 22 ) 

and that 

(23) X*.x = • Px • . 

By substituting (22) into the expression for Zn{x) in terms of one can 
show that 


(24) 


Zn{x) = t 


= Exrx‘($..x-i)(i-'*'..x). 


Therefore Z„(a;) can be determined from a knowledge of the characteristic vectors 
and values of the matrix P* . 

If there exists a largest characteristic root X,,* such that 

(25) Xl.x ^ 1 Xt,x 1 if i 7 *^ L, 

one can obtain some interesting results. Before deriving these, we shall give a 
sufficient condition (which is satisfied in many chains) for the existance of this 
inequality. Frobenius [11] has shown that if all the elements of a finite matrix 
are > 0, then the characteristic value of largest absolute value of the matrix is 
real, positive, and simple (nondegenerate). Thus, as long as p is finite and 
P*(«, P) > 0 for all a and jS, (25) is valid. 

We shall now prove that 


(25a) 
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that is, 

(25b) Z„(a:) ~ • 1)(1 • 


First let us consider the case in which P, is a symmetrical matrix. 
(Pj.xM == all the characteristic values are real, and 

Znix) = 

if^L 

From Cauchy’s inequality and (21) 


Therefore, 




22 »’<.*(«) 

a-l 


^ ij = 


Ex.^rWi)’ Zxr,r‘ < k.'- dix.t’i 

if^L xf^L 


Then 


where X,,* is the characteristic value of second largest in absolute value. 
This inequality yields 


(25c) 


Znix) 

x,!-‘($.,.-l)= 


1 


v(v - 1 ) I Y ‘ I 

- (4>/...-1)MVxW I 


and (25a) (since \>a/h.,x < 1) follows, 
easily derive the analogous expression 


Zn ( a ) 

xz:;‘(4>i,.-i)(i 


When Px is not symmetrical, one can 
A(v -- 1)1 WXL. xh^^ 

“■ (< pL , x - l )( l *^/., x ) 


where 


A = [max (I • 1) inirnax {| (1 • ^».x)|)] 


For brevity, when a; = 0, we write 'Kt,z as X,, as and as . By 
summing (10) over all a’s except ai , a* and an we obtain the probability of an 
intermediate event leading to a result a* if the results of the first and last events 
are known to have been ai and a„ . With the aid of (21) and (22) it is easy to 
show that this probability is exactly: 


(26) 


y 

i.i-1 


Ex?-* E i^.(«i)^.(«») 

*-l 


When n is very large, and when we have simultaneously n > > A; > > 1, we can 
rewrite this equation to include Xl , and neglect all terms containing other t’s alld 
i's. This leads to the results 

a) If the number of events, n, in a simple chain is very large, the probability 
Pnioth) of a A;th event far removed from the first and the last, 3 delding a 
result ak when ai , and an are unspecified is 

Pn(otk) ^ ^L(ak) (pdoik) / {^L • 1)(1 • ^ l ). 


(27) 
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b) When k = n, the probability of the result ai • of the first event leading to 
the result a* of the nth event is 


(28a) 

So, as n —> 00 
(28b) 


23 V»(ai)v>i(«n) 

Pn(<Xu (Xn) = 


Exr-‘ Z 

*-l ttl.ttn 


^n(«l , «n) 


^L(ai)0L(an) 


c) When there e^dsts no knowledge concerning the result of the first event, the 
probability of the nth event yielding the result «« is 

(29) Pn(otn) = 23 Pn(otl, «„) ^i:.(an)/(l * 4 >l) . 

In chains of sufficient length for (25) to be valid, the probability of 

F(ai ,•••,«„) 

having a value between { and i + h has an especially simple asymptotic form. 
From (6) this probability is (if for a given n we let T = an*) 

/j \ 

—Ai) 


(30) 


G(f + h) - G(J) = ^. lim j ' (^) I 

2Tn a-*00 \ w / 


(1 - e'*"*) exp^ -Jco'^Ai - 


and from (25) and (5) 
(31) 
if 


n lim a”* log }iL,x/dx^ = nLm 
*-*o 


31 


+ 


(32) 

Lm = lim a”* log 
*-*0 


Ijctting y = an*, 

(30) becomes 


(?(? + h) - 
(33) 

G({)~^.lim /“^ 

ZtTtl a-^oo J—a y 

Uy^i , "I 

6n» / 

where 



(34a) 

Ml = (€ — Ai)/n* 

M 2 = (f + A — Ai)/n* 


(31b) 

Ai = average value of F(ai, • • • , 
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Integrating (33) 

(35) G(i + h)- Gii) ~ + 0(l/n)l d^. 

As n —> 00 and h ^ 0 

(35a) G(? + h)- G(f) ~ exp (-i)[| - F]/nL^), 

and the probability that i < F < ^ + h becomes Gaussian. 

b. Examples of a simple chain. As an example of a simple Markoff chain let 
us consider an event which can lead to either of two possible results, say —1’* 
or Further, let us suppose that the probability of a given result being 

followed by an identical one is p and by one of another type is (1 — p); that is, 

p(-l, -1) = p(l, 1) = p 

p(-l, 1) = p(l, -1) = 1 - p. 

This chain would be encountered in an analysis of a sequence of tosses of a 
coin with a “memory” so that the probability of two successive tosses showing 
the same face of the coin would be p and that of showing opposite faces (1 — p). 

A question one might ask concerning such a chain is—What is the probability 
of the occurrence of a given number of transitions from one kind of result to 
another? In the chain of results 


- 1 , - 1 , - 1 , 1 , 1 , - 1 , 1 , - 1 , - 1 , -1 

there would be four transitions, one corresponding to each — 1 followed by a 1 
and to each 1 followed by a — 1. The function giving the number of transitions 
in a sequence of n events is 

n—1 

(36) F(ai, •••,««) = h(ai , a<+i) 

where 

M~l,-1) 1) =0 

1) = Ml, -1) = 1. 

Even though the a’s are dependent, in this special case, h(ax, a,+i) and 
h(at+i, ai+i) are independent so that (40) could have been obtained on this basis. 

To apply the methods described in the beginning of this section we must find 
the characteristic values and vectors of the matrix 


(37) 


/ P (1 - P)e^\ 

\(1 - p)e* p / 


(the configuration index a has the value either — 1 or 1 in this case instead of 
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‘*1” and “2” as given in (17)). The characteristic values are the roots of the 
equation 

P - X “ P)«' _ Q 
(1 - p)e* p - X 

that is, 

(38) X,.. = p + (1 - p)e" 

I Xj.x I = 1 p - (1 - p)e* I < X,,, 
and the characteristic vectors are 

^..x = 2-* and = 2-^ . 

The ^ and vectors have the same components in this case because of the sym 
metry of the Px matrix. Clearly 

X£, = Xi = Xi,o = 1 > X 2 = X2,o ~ 2p 1 

^i(a) = 2“^ and ^ 2 (a) = —a • 2“^. 

From (26) we see that if the result of the first event in the chain is ai , and 
that of the wth event is an , the probability of the /cth event yielding the result 
a* is 

[(2p — l)^~^aiajfc + 1][1 + (2p — l)”~*a;fca„] 

2[1 + (2p ~ l)"-iaianl 

As /c, Ui and (r? — k) simultaneously get very large, Pn(«A) 2 > independently 
of ak. 

The probability of an initial result ai leading to a final result is (from 28a) 

Pn(«i , «n) = (!){! + (2p — 1)”'^ ajan) 

so that 

Pn( 1, 1) = P.(-l, ~1) = (i) {1 + (2p ~ 1)'*-^! 

Pn(-1, 1) = Pn( 1, ~1) = (1) {1 - (2p - 1)”-^}. 

Now, to answer our original question regarding the probability distribution 
of the transition function (36) 

n-l 

(39) ‘ F{oti , • • • , an) = /i(a., a<+i), 

we use the expression for Zn(x) determined from (24) 

(39) Z,{x) = 2[p + (1 - p)e*l"-‘ 
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From (9) the probability of there being between f and i + h transitions in a 
sequence of n + 1 events is 


(40) 


(?(€ + h)- G(f) = ^ + (1 - p)e*T du,/o> 


_ 1 f ( -M _ -Mi-hu - p) p' 

2«1. ^ (n-k)\kl * 

Letting x — uh/2 and rearranging 

at, + » - am + «). 

where Z)(X) is the Dii-ichlet integral 


D(X)=1 = i 

"K J—tc X 


0 if lx| > 1 

1X1 = 1 
1 |X1<1. 


We therefore have, when [{ + h] < re 
(41) 


fa-lt+il (re — «}!k! 


Here [x] denotes the greatest integer not exceeding x. The sum is zero if 
[{ + h] < [{ + !]• When [f + ft] > re 


(42) 


<!(! + W-0(«- t 


When n is large it is difficult to get a clear picture of the function (/(f) from 
(41) and (42), so we shall develop asymptotic results for large n by using (6) 
instead of (9). 

By emplo 3 dng (5), we see that (this section will be developed on the basis of 
n + I trials instead of n) 


Ai = P = n(l — p) 


A 2 = np(l - p) 

As = np(l — p)(2p — 1) etc. 


Therefore, from (6) 


AG = G(f+ « - G(€) = i jf 


00 —»«({—Ai) 


(1 - e-*“*) 


exp [—Jnp(l — p)«^ — inp(l — p)(2p — 1)«V0 — • • •] dta. 
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Letting u — we have 

^ ^ f •u({—Aj)/n^ ^-~iu($-hh--Ai)/nii 

27n J—ao u 


-iu^pU~p) 


uyLy-'-'b 


tp(l - p)(2p - 1)m’ 


6n* 




where 


Ml- = (J + — Ai)/n* 

M2 = (f ~ Ai)//i*. 

Since 

f “c -““’€ •*■'“ dw = (x/o)‘ exp (-xV4o) 

J—00 

we have for large ri 


AG 


1 


(43a) 


[2irp(l - p)]i 


/ M2 

^-X2/2p(l-p) 
ui 




As ?i —► oc and /t 


\* 2p(l 
' 0, this becomes 


G(J + h) - G(0 


(43b) 


^ exp ! - [| - ff/2p(l - p)wi 
[2irnp(l - p)]t 


1 


(2p - .DJLIjP) 
2p(l — p)n 


+ 


»©) 


A similar problem which occurs in statistics of high polymers can be stated 
abstractly as follows. Suppose there exists a sequence of events each of which 
leads to a translation of length a of a point either to the right or to the left, and 
that the probability of a translation continuing in the same direction as its 
predecessor is p while that of changing its direction is (1 — p). After n trans¬ 
lations what is the probability of a point being displaced a distance { from its 
origin. 

If ‘‘-r' represents a translation to the left and “+1” a translation to the 
right, 


p(-l, -1) = p(l, 1) = p 
p(-l, 1) = p(l, -1) = (1 ~ p) 
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The function giving the distance of the point from its origin after n displacements 
is (when a = zfcl) 

n 

F(ai, • • ■ , a„) = = ioai + h(ai , a 2 ) + • • • 4- , a») + \aot^ 

where 


Ml, 1) = a, M~l, -1) = -a 
Ml,-1) =M~1,1) =0. 

Neglecting the terms aai/2 and aan/2 in F{ai ,••*,«„), one can answer questions 
concerning this problem by evaluating Zn{x) as defined by (15). In this case 
Px has the form 


P. = 




Its characteristic roots are 

Xi.a = V cosh ax + cosh^ oa; + (1 — 2p)]* = Xl.® 

I X 2 .* I = I P cosh ax — [p^ cosh* or + (1 — 2p)]* | < Xi.a:. 


and its characteristic vectors: 


= [(p - D’ + (pc~ - xj)r* 


4'i.x — [(p ~ 1)^ + (p®”* — ^ 2 )*] * 



Since 

P = Ai = lim d log Zn{x)ldx, 

*-*0 

one can show in the present problem that P = 0. Therefore, the probability 
of the translated point being a distance between f and f + ^ from the origin 
after (n + 1) translations, is, as n 00 and A —> 0 

+ h) - ii’(J) ~ fe(22r«L2)'*c"*’'*"'" 


where La is by (32): 

La = lim d* log \l,x/^x = o*p/(l — p). 

x-O 

Thus, 

F({ + h) - F{& ~ h[a^imp/{l - 

When p = 2/3 this problem is equivalent to the determination of the proba- 
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bility distribution of the components in an arbitrary direction of the distance 
between the ends of a linear polymer. In this case 

F(S + h) - F(i) ~ A(4o*im)"* exp (-{’/W) 

a result obtained by Tobolsky [12] after a lengthy and complicated combinatory 
calculation. 

Another type of simple chain is encountered in the determination of the 
“life span*’ of a particle which is displaced a unit distance to the right or left 
per unit time along a straight line until it collides with an absorbing boundary 
either — (3 + 1 ) or (p + 1 ) units from the starting point. This problem has 
been analyzed by M. Kac using the methods discussed in the present paper. 
We shall generalize his results to include the effect of an attraction of the particle 
toward one end of the line so that displacements toward that end are more 
probable than those in the other direction. 

Following the notation of Kac [13] we let Xj represent the jth displacement, 
my its length, and 6 (m) the probability of a given displacement having the 
length m. Then, 

s if m = 1 

5(m) = 1 — 5 ifm =—1 

0 otherwise. 


If N represents the life span of a particle, the probability of its exceeding n is 
Prob {N > n} = Prob {—g < Xi < p, —g < Xi + < P, • * * , 

—5 + -^2 + • • * + Xn ^ p 1 = S 8{7ni)8{mi) • • • 8{mn) 

where the summation extends over all integers mi, m 2 , • • • , m,, such that 
—q < mi < p, —g < mi + 7n2 < p, • • • , —(7 < mi + m 2 + • • • + m„ < p. 
Defining the new set of variables 

ay = g + mi + wi 2 + • • • + my 0* = 1, 2, • • • n) 

we see that 

P+q 

Prob [N > n] = X) 5(«i — aO • • • 6(a„ — otn-i). 

As before, if we introduce the P matrix (of p + g + 1 rows and columns) 


P = (5(a - P)) 


0 1 - 5 0 0 

5 0 1 — 5 0 

0 5 0 1—5 
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we obtain after applying the equivalent of (22) 

Prob {N > n] = ^ 

J—1 aii—0 

Where \j is the jth characteristic value of P, and ypj and <pj are its associated 
characteristic vectors as defined by (19) and (20) (here the range of a starts 
from 0 instead of 1 as in (17) and (19)). 

It is easy to show that the characteristic values of P are 

X/ = 2[s(l - s)]Wfj 0* = 1, 2, • • • , p + 5 + 1) 

where 


f/ — v/ip + g + 2) 

and that the components of the characteristic vectors are 

= [2/(p + g + 2)]*[s/(l - s)]‘“ sin (a + l)j-, (a = 0,1, • • • , p + g) 
and 

<p,{a) = (2/(p + g + 2)]*[(1 - s)/s]‘“ sin (a + l)f,. 

Since 


^ _ V2 (1 - 8) {1 - sinf, 

Vp + g + 2 1 - 2[8(r - s)]» cos f, 


we finally have 
Prob {N > n] 


p + g + 2 

jl - (-1)^(8/! - s)**”'*'"'*'”} cos" j-, sin sin (g + l)f, 
1 — 2^8(1 — 8) cos f, 


When 8 = J this reduces to the result of Kac: (* means summation is only over 
event’s 


Prob {N > n} 


n p+Q+1 

—-—23 * cos" f, an (g + l)f, cot if,. 
p + g + 2 ,_i 


4. Simple Chains with Restrictions. Often when studying chains of dependent 
events, certain functions averaged over the entire chains are known to be 
restricted between definite limits. That is, there might exist k functions 
0j(oci , Q !2 , * * * , cifn) such that 

(44) —AG; < G; — gj(ai ,•••,«») < AG/, (; = 1, 2, • • • fc), 

where the Gj’s and AG;^s are preassigned constants. To calculate averages of 
other functions (1) is no longer valid, for it is an unrestricted sum over all sets 
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of a% including those incompatible with (44). All unrestricted sums in this 
formula (and other similar ones) must be replaced by sums over only those 
sets of a^s compatible with (44). Since it is sometimes more difficult to evaluate 
restricted sums than unrestricted ones, we shall apply an idea of Markoff [ 1 ] 
to the reduction of the former to the latter type. 

Let us seek an explicit expression for a function Pt(ai , a 2 , * * * , «„) which 
has the property: 

P^(«i, • • * yCtn) = Pnidi, • • • , a„) when a’s are chosen 

so that (44) is satis¬ 
fied of all j. 

0 otherwise. 

Since the Dirichlet integrals 

1 /*" sin (pyAG,) V , 

Sj = - - 2 - L exp (zpi 7 ,) dp, 

TT J-oo P/ 

have the property 

5y = 1 when —AGy < 7 / < AGy 
0 otherwise, 

P*n{cLi , • • • , an) = 5 i52 • • • hPn{a \, • • • , an) 
has the required character provided 

yj = Gj gj(ai , • • • , aj. 

The average value of a function F{ai , • • • , an) can be written in terms of the 
unrestricted sum 

^ P(ai , • • • , an)Pn(«l , • • * , OLn)/ Pn(«l , * * * , ««), 

!«•} !«•) 

where the summation extends over the complete set of {a,} 

{a,} = (ai, a 2 , • • • , an). 


As in the case of chains without auxiliary restrictions, a useful function is 
Zn{x) = Ptiai an) exp {xF{ai , • • • , an)) 

= A ■ r PI, • • •. p*) n dp J 

TT* J—Qo »»—1 Pm j 

where 

SniXy Pi, • • • , P*) '== Pn(«l , • • * , an) 




exp |a:F(o,, • • •, a.) - 12 P/P#(ai, • • • , a»)| • 
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When F{ai and {gj{ai , • * • , an) 1 are all additive or multiplicative 

functions of the form (13a) and (13b), say 

n-l 

F{ai , • • • , an) = 23 Hotk , a*+i) 

ib-l 


gj(oti , • • • , a„) = 23 9j(^k , afc+i) 


and the probability chain is a simple one, Zn{x) reduces to a simple form. 
Suppose 

n—l 

Fnioti , * • * , an) = 23 Pi^J 1 ^?-fl) 

;-l 

then following the derivation of (24), we have 


(46) 


Sn(x, Pi, ■ ■ ■ , Pk) = T, {X,..,.r~‘(4>,..,p-l)(l-4',,.„) 


where Xj,.,,, 4>j,x,f and are characteristic values and vectors of the matrix 

/Px.p(l, 1) • • • Pi,p(l, >')\ 


Px.. = 


Vpx,p(>', 1) 


Px,v)) 


and 


Px.(.(a, 0) = P(a, /3) exp \xh{a, P) - i Pi9,io‘7 


Substitution of (46) into (45) allows one to calculate Zn{x). 


5. More Complicated Chains. In a chain of N events in which the result of 
each event depends on those of its n predecessors (n < < N), the calculation of 
Zn(x) proceeds in essentially the same manner as in the case of a simple chain. 
Let the N events be divided into N/n sets of ^‘grand events^’ of n simple events 
each (for simplicity we assume N is divisible by n, this can easily be avoided). 
Thus, if each simple event could lead to any one of v possible results, a grand 
event could lead to anyone of possible results and a complicated chain becomes 
a simple chain of grand events with the result of each grand event depending on 
the preceeding grand event. Quantitative calculations thus proceed formally in 
the same manner as in a simple chain. 

6. Continuous Case. In this section we generalize, by studying an example, 
to the case in which each event in a simple chain may lead to any one of a con¬ 
tinuum of results. The example is a problem arising in statistical mechanics of 
molecular chains. 

Consider a linear chain of n identical molecules whose centers of mass remain 
at a set of fixed regularly spaced positions, but which may rotate about their 
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centers of mass in a plane. Suppose, that the potential energy of interaction 
between neighboring pairs of molecules is a function of the angles a specified 
axis of the molecules makes with the line connection the centers of mass of the 
molecules; that is, the potential energy of interaction between pairs of adjacent 
molecules can be written as Bj+i). Assuming that forces are sufficiently 

short ranged for interaction between more distant neighbors can be neglected, 
Boltzmann's theorem states that the probability of the axis of the first molecule 
making an angle between di and di + aBi with the line of centers of the chain, 
the second l>etween B^ and B 2 + 01 B 2 and the nth between Bn and Bn + dBn is 
proportional to 

exp {-‘kT , ^ 2 ) + , ^ 3 ) + * * * + V{Bn -\, ^n))] dBi • • • dBn 


where k is Boltzmann's constant and T is the absolute temperature. The 
contribution of the interaction to the thermodynamic properties of the chain 
can be derived from the partition function 


(47) 


n 2v 

. ‘”1 






For example, the internal energy is 

E = aiogZ„M-iAT) 


and the specific heat is c = dE/dT. 

It is to be noted that Z„ is exactly the integral of the iterated kernel of the 
integral equation 

(48) Midi) = I ^(02) exp , 02) I d02. 


If V{6i , B 2 ) is symmetrical in Bi and 02 , this linear homogeneous integral equation 
has a set of orthonormal characteristic functions {^,(0) i such that 


(49) 


Jo 


dB = 8 jk • 


To each of these characteristic functions there corresponds a characteristic value 
Xy. Now it is well known that the kernel of (48) can be expanded as a series in 
its characteristic functions 

exp V(e,, 02)| = Z X, UeOUOi). 

Introduction of this expression into (47) and applying the orthogonality condi* 
tions (49) one obtains 

Zn = Zxr‘{f 


(47a) 
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Probably the most interesting example of a molecular chain of the type 
described above is a chain of magnetic dipoles which are restricted to rotate only 
in a plane. In that case 

2 

= ~[C0S (Bf - ^y+i) - 3 cos Bj cos ^y+i]. 

Where /i is the magnetic moment of each dipole and r is the distance between a 
pair of adjacent centers of mass. This potential function leads to the integral 
equation 

\\l/(Bi) = jf \p(B 2 ) exp ^;^[cos (Bi - ^ 2 ) ~ 3 cos Bi cos ^ 2 ]| . 

Since this equation is rather complicated to solve, we shall devote the rest of the 
section to a potential function of less physical interest, but which leads to a less 
formidable integral equation. 

In studjdng hindered rotation of molecules, one sometimes uses potential 
functions of the form: 


^/+i) = -0 cos {Bj - 0y+i) 

where /S is a constant. With this potential function (48) becomes 
(50) X^(^i) = / ^(^ 2 ) exp {J cos (Bi - ^ 2 )! dB 2 

Jo 

where 


J = /8//cT. 

The characteristic functions and characteristic values of (50) are easily found 
with the aid of the Fourier Series for exp (J cos B): 

eo 

(51) exp (J cos 9) = h{J) + 2 23 Im(J) cos m 9 


where Im{J) is the mth Bessel function of imaginary argument: 

(i J)"'"” 


UJ) = Z 


SS. (m + k)lk\ 


From (51) 

00 

exp 17 cos {9i - ^ 2 )] = /o(7) + 223 Im(7)(cos m9i cos mB^ + sin m9i sin mBi). 

rnmml 

Substituting this expression into (50) we have 

MiOi) = r lA(^») 0 ( 7 ) + 2 23 /■»(7)(cos m9i cos m9t + sin rtiBi sin mSt) | d9i. 
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Because of the orthogonality of the trigonometric functions, one can verify by 
direct substitution that the characteristic functions are 

Ue) = l/(2ir)* 

sin m0; cos mO, (m = 1 , 2 , • • •) 

and the corresponding characteristic values are 
Xo = 

= 2tIM) m > 0. 

Introduction of these characteristic functions and values into (47a) we obtain 
the simple formula for the partition function: 

Zn = 2irl27rIoiJ)r-\ 

The internal energy of the molecular chain is therefore 
E = aiogZn/a(-i//bT) 

= ~/3(n - 1) UJ)/W), 

and the specific heat is: 

(7 - ai»T . »(»- iw{i + . 
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ON THE FIRST TWO MOMENTS OF THE MEASURE OF A 
RANDOM SET 

By L. a. Santal6 

Universidad Nacional del lAtoral, Argentina 

1. Introduction. In a recent paper [3] H. E. Robbins derived general formulas 
for the moments of the measure of any random set X, and applied the formulas 
to find the mean and the variance of a random sum of intervals on a line. In 
subsequent papers, J. Bronowski and J. Neyman [1], using other methods, found 
the variance when X is a random sum of rectangles in the plane, and H. E. 
Robbins [4] found the variance when X is a random sum of n-dimensional 
intervals in n-dimensional euclidean space. In the latter paper Robbins 
solved also the corresponding problem for circles on the plane. 

Using the methods of Robbins, our purpose in the present paper is to solve the 
following similar problems: 

(i) Let R denote the rectangle consisting of all points {x,y) such that 0 < x < Ai, 

0 < y < A 2 , and let R' denote the larger rectangle for which --5<a:<Ai + 3, 
— S < ^ < A 2 + 3. Let p denote a rectangle of fixed dimensions, a X 6, but 
variable position in the plane. The position of p will be determined by the 
coordinates or, y of its center P and the angle ip between the side of length a and 
the .r-axis. We suppose (a^ + < min {Ai , ^42,6). Let a fixed number N of 

rectangles p be chosen independently with the probability density function for 
the coordinates (a*, y, ip) of each rectangle constant and equal io \ ir R* in the 
three-dimensional interval with base R' and height tt and zero outside this 
interval. In section 3 we evaluate the first two moments of the measure of X, 
where X denotes the intersection of the set-theoretical sum of the N rectangles 
p with R, 

(ii) Let R denote the ?i-dimensional interval consisting of all points (xi , , 

•Ta , • • • , -Tn) such that 0 < a*. < ^ , (i = 1,2, • • • ,n), and let R' denote the 
larger interval for which — 5 < x, < .1, + 5. Let a fixtnl number N of w-dimen- 
sional spheres with radii r (such that 2r < min (A, , 2 5)) be chosen independently, 
with the probability density function for the centre of each n-sphere constant 
and equal to \/R^ in /?' and zero outside this interval. Denoting by X the 
intersection of the set theoretical sum of the N n-spheres with 7?, we evaluate 
in section 4 the first two moments of the measure of X^. This problem is a 
generalization to n-dimensional space of the case considered by Robbins for the 
plane (n = 2) in [4]. 

2. Preliminary formulas. Let K be an indeformable plane convex figure of 
variable position in the plane. The position of K may be determined by the 
coordinates (x, y) of a point P fixed within K and the angle ip which measures 
the rotation of K about P. We shall call x, y, tp the coordinates of X. The 
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measure of a set of figures congruent with K is defined as being the integral of the 
differential form 

(2.1) dK = dxdyd<p. 

It is readily shown that this measure does not depend on the particular point P 
chosen to determined the position of XfS]. For instance, the measure of the 
set of figures K, each of which contains in its interior a fixed point Q, has the 
value 2 ttF, where F denotes the area of K ; that is, 

(2.2) f dK = 2irF. 

jQtK 

Let Pi and P 2 be two fixed points and let I be the distance P 1 P 2 . The measure 
of the set of figures congnient with K, each of which contains both points Pi 
and P 2 in its interior, will be a function of K and /, say yiK, 1). If d is the 
diameter of that is, the maximal distance between two points of jfiC, we have 
Z) = 0 for Z > d. 

Examples. Let K be a rectangle p of fixed dimensions a X h, and let us 
suppose a < h. The diameter d of p is d = (a* 4“ Z>^)*. Let P(a;, y) be the 
centre of p and <p the angle which forms the side of length h with the segment 
line P 1 P 2 of length Z. If we keep first <p constant, then in order that there exist 
positions of p in which it contains the segment line P 1 P 2 in its interior it is neces¬ 
sary that 

a — Z sin ^ > 0, 6 — Z cos ^ > 0 

and in this case the area covered by the centres P in all these positions has the 
value 


(a — Z sin <f>) (b — I cos <p). 

Integrating over all permissible values of we obtain 


(2.3) m(p, 0 = 4 /' 

•ai 

where we define 


>aro8in[a/ni 


arcco8[&/ni 


(a — I sin ip)(b — Z cos ^p) dtp 


Mi = 


a; if a; < 1 


1 if a; > 1. 

Carrying out the obvious integration in (2.3) we have 

f 2 7ra6 - 4 Z(a + W + 2 Z' for Z < a < 6 

' 4(ab arc sin (a/Z) — i — bl + b{f — a^)^) 

(2.4) p(p, Z) = ^ for o < Z < 6 

1 4(a6 arc sin (a/Z) — arc cos (6/Z) + bif — a*)* 

^ + o(Z* - 6*)* - i(a' + b^) - i f) for a < 6 < Z. 
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As another example, let R be the rectangle consisting of all points (x, y) such 
that 0 <a:<Ai, 0 <i/<A 2 and let be the rectangle consisting of all points 
{x, y) such that 

-d < X < Ai + 8, -6 < y < A 2 + d, (a' + bV < min (Ai, A 2 , 6). 

Let us consider the set of rectangles p whose centers belong to /2' and do not 
contain either Pi or P 2 , Pi and P 2 being two fixed points which belong to R, 
Let I be the distance P 1 P 2 . According to (2.2) and the definition of /x(p, 1 ) 
the measure of the set of rectangles p under consideration is 

(2.5) 2 ttR' - 2.2 wp + /i(p, 0, 

where R' = (Ai + 2 8 ) {A 2 + 2 5) and p = ab. 

Let K be a plane convex figure of fixed position in its plane. Let us suppose 
K to be translated a distance I in the direction 0, and let F(Km, I, 6) be the area 
of the intersection of K with the ti’anslated figure. Obviously if d is the diameter 
of K, F{K, I, 6) = 0 for I > d. In what follows we shall consider the function 

-2r 

(2.6) HK, I) = / F(K, I, e) de. 

Jo 

Example. Let X be a rectangle R of sides Ai, A 2 . Let the s 5 nmbol [x], as 
in [1], be defined by 

x if X > 0 

[x] = 

0 if x < 0. 

It is then readily seen that 

(2.7) F{R, I, 6) = Ml — Z sin B] [A 2 — I cos B], 

For our purpose the case in which I < min (Ai, + 2 ) is of interest. In this case, 
carrying out the immediate integrations, we obtain 

(2.8) -*>(/?, 0 = 2 T - 4 Z(A, + A 2 ) + 2 l\ 

Let Sn.r be an n-dimensional sphere of radius r. S„,, will denote also the 
volume of this sphere, that is, as is known, (see [2, p. 109]), 

(2.9) ■’ r(" + i)' 

Let us call the measure of a set of spheres S.., the measure of the set of their 
centers. That is, if the point P(xi , X 2 , • • • , a;») is the center of Sn.r the measure 
of a set of spheres Sn.r equals the integral extended over the set, of the differential 
form 


(2.10) 


dP = dxidxt • • • dXn . 
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For instance, the measure of the set of spheres aSw,, , each of which contains a 
fixed point Q in its interior, has the value 

( 2 . 11 ) [ dP ^ Sn,r 
where Sn,r is given by (2.9). 

The measure n{Sn,r , 0 of the set of spheres Sn,r , each of which contains 
totally in its interior a segment of length /(/ < 2 r), equals the volume of the 
intersection of two-spheres Sn,r whose centers are placed at the end points of the 
given segment. That is, ti(Sn.r , 1) equals twice the volume of the spherical 
segment of an n-sphere of radius r and semiangle a = arc cos {l/2r). We will 
represent the volume of this spherical segment by and it maybe calculated 

in the following way: The intersection of the n-sphere with a hyperplane at a 
distance x from the center is an (n — 1 )-dimensional sphere of radius r' = 
(r^ — x^)K Let /Sn-i.r' denote the volume of this (n — 1 )-dimensional sphere 
(given by the general formula (2.9)). The volume of the spherical segment,, 
whose base has the radius h = r cos ct, will be 

~ l,r' dx. 

Putting X = r cos d and substituting for Sn^i,r^ the expression given in (2.9), 
we obtain 

l)/2y.n pa 

¥r 

Consequently we can write 

( 2 . 12 ) n{Sn,r , 1) = 2SnAcc) = r sin” 6 dd, 

Jo 

where Sn~i.r is the volume of the (?i — 1 )-dimensional sphere of radius r and 
a = arc cos (i/ 2 r). 

In (2.12) we may substitute 


AS'fl,r(a) = 


sin" 6 dd 


= rSr.-l.rf 
Jo 


sin" 6 dd. 


/■«n- 

Jo 


Odd = 


- l)(n - 

3) • • • 3.1 

rp 

cos 

1 (l/2r^ 

n(n — 2) . 

..4.2 

1 ypi^Tj 


^2 \ (n-l)/2 

_L. 

(n - 1) 

2r \ 

" 4r7 


n(n — 2) 

4- . . . -L 

(n ~ l)(n — 

3) 

...3.1 / 

1 1 

n(n — 2)- 


4.2 V 


j2 \ (n-3)/2 


(2.13) 
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for n even, and 


(2.14) 


fsin") 

•'0 




(n - l)(w - 3) ••• 4.2 
n(n — 2) ... 3 


2r\n\ 4r®/ 


, n-l ( _XY" - l)(w - 3) • •. 4.2 '1 

n{n - 2) \ 4r2/ n(n - 2) ... 5.3 / 


for n odd. 

In particular, for n = 2,3 we have 


(2.15) iJL(S 2 ,r,l) — 4r* f sin^^ d6 = 2r^arc cos (l/2r) — \ 

Jo 2 

(2.10) n{Si,rX) = 27rr* f sin0 d0 = ^ + — xZ^. 

JO 3 12 


We shall now generalize the formula (2.8) to n-space. 

A direction in n-space may be given by the corresponding point on the surface 
of the n-dimensional sphere of unit radius, that is, by the end point of the radius 
which is parallel to the given direction. The parametric equations of the 

n 

n-spherc {! = 1 are 
1 

11 = cos <pi 

^2 ” sin (pi cos ip 2 

(2.17) fs = sin ipi sin ^2 cos v ?3 


^n —1 — sin ipi sin ^ • sin ^n —2 cos ^n —1 

f« = sin ipi sin <p 2 • • • sin v?n -2 sin ^„_i, 

where 0 < ^» < x for f < n — 1 and 0 < (pn-i < 2 x. The element of area of 
this n-sphere has the value (see, [2, p. 109]) 

(2.18) da = sin""”^ <pi sin"“^ <^2 • • • sin tpn^z d^pidip^ • • • d^n-i. 

A direction in 7t-dimensional space may then be given by the n — 1 parameters 

Given the w-dimensional interval R consisting of all points (a:i, 0 : 2 , Xs, * * • , x„) 
such that 0 < X, < A* {i = 1, 2, 3, • • • n), and suppose that R is translated a 
distance Z(Z < min (Ai, A 2 , A 3 , • • • , A„)) in the direction (^ , ^ , • • • , ^n-i), 
the intersection of the translated interval with R is& new interval whose volume 

n 

has the value 11 (^* “" where Xi = Zf< (J» given by (2.17)). 
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Our purpose is to evaluate the integral 


(2.19) 


m,l)=f il(Ai-Xi)d,T 
Jgn 1 


extended over the surface E„ of the n-dimensional sphere of radius unity. We 
shall denote by Em either the surface of the m-dimensional sphere of radius unity 
or its area, given, as is known [2, p. 110] by 


( 2 . 20 ) 




Ki)' 


Because of the symmetry, the coefficients of all the products • • • A,-,.* 

have the same value 


Otk 



XxX 2 • • • Xk d<r. 


The integral extended over tlie whole surface En equals 2" times the integral 
extended over the portion for which > 0. Plence, taking into account (2.17) 
and (2.18) we get 

^»/2 A)r/2 

ofjb = (—1 )*2*^^^n--jfc / *** / sin”^* cossin”^* \p2COSv52 

Jo Jo 

(2.21) • • • sin” ^ Va: cos ipk dipi dUp^ • • • dpk 


^ _ ^l^En-k _ 

^ ^ {n + k- 2)(n + k - A) • • • {n + k - 2k) 

£or A: = 1, 2, • • • , n — 1. For k = nwe find that 

prl2 pwl2 

Qfn = ( —l)”2”r / ••• / sin^"“Vicosi^i 

Jo Jo 


( 2 . 22 ) 


• • • sin <pn-i cos ¥>»-! dipi dip 2 • • • d<pn~i 


= (- 1 )" 


2T 

(2n - 2)(2n - 4) 


• 4.2 ■ 


Hence, we have the following general formula 


HR,l) =AxAi--- A„E. +(- 1 )" 


2T 

(2n - 2)(2n - 4) ... 4.2 


+ i: (-!)*( L A,,Ax,---Ax,.,) 

fc-1 n.*2.* 

•21‘fEn-k 

{n + k- 2){n + fc - 4) ... (n + fc - 2A:)' 


(2.23) 
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In particular, forn = 2 this result coincides with (2.8). For n = 3 we have 


(2.24) 


$(i?, 1) = ArAiAiA, -f- 2rl(AiA2 + AiAz + A^A,) 

+ ll\Ai + + At). 


3. First problem. We can now solve the first problem (i) stated in the intro¬ 
duction. Denoting by the same letters either sets or their measures, we consider, 
as in [1] and [4], the set Y of points of R that do not belong to X. We have 
identically: 

(3.1) X+Y = R. 

The general method of Robbins [3] taking into account (2.2), gives immediately 
tbe first moments 

(3.2) E(Y) = r(i- , E(X) = R |l - , 

where R = A 1 A 2 = (^1 + 25) (^2 + 25), p = ah. 

Our remaining problem is that of evaluating the second moment of X. I^et 
a*i, Vi , = 1, 2,3, • • • , N) be the coordinates of the N rectangles p (section 2) 

and let us put, as in (2.1), dp< = dx4y4<pi . Let P{x, y) and Po{xo , 2 / 0 ) be two 
points which belong to R and let us put dP = dx dy, dPo = . Let us 

consider the following multiple integral 


(3.3) 


/ dP dPo dpi dp2 * • • dpAT 

(2^PT 


extended over the sets of rectangles p»- (congruent with p) such that , yi belongs 
to R\ 0 < <pi < 27r, and do not contain either P or Fo. That is, the domain of 
integration of J is defined by 


(3.4) 


— 5 < Xt < Ai + 5, < t/i < i42 + 5, 0 < > 2 t , 

P € R, PoeR, Pi Pi, Poipi, {i = 1,2, - • ,N). 


In order to calculate J, we can first keep the rectangles pi fixed; the points P 
and Po can then vary independently over the set of points Y. That gives 

(3« J- I 

(xt.yi)A’R' 

We can now reverse the order of integration, an operation which is obviously 
justified in this case. Keeping P and Po fixed, we can vary each rectangle p< 
over the set of positions in which it does not contain either P or Po ; letting I 
denote the distance PPo , we have, according to (2.5), 

(3.6) J= j 

P€R,Pq€R 
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In order to evaluate this integral we divide it into two parts / = » 

accordingasO<2<dord<i<Z>,whered = (a^ + and D = (A? + A 2 ) . 
In the interval 0 < Z < d we introduce the new variables of integration L B 
related to x, y, Xq , yo by 

(3.7) xo == X I cos Bf yo = y + Isin B 

whence 


a( x, y,XQ, yo) ^ ^ 

d(x, y, /, B) 

In terms of the new variables we have 


In this integral the point P can vary over the intersection of R with the figure 
obtained by translating R a distance I in the direction 6; that is, the integration 
of dP gives the function F(/?, Z, 9) defined in section 2. According to (2.6) we 
therefore have 


(3.8) - r (' - 

where /h(p, 1) is given by (2.4) and <i>(/iZ, 1) by (2.8). 

In order to evaluate J 2 we observe that in the interval d < I < D nip, /) = 0 
and we have 


J2 =(1 - f dP dp ^=- |?y r Y dP dPo - j dP dP, 

^ J 0' I 


Further we have 


(3.9) I dP dPo = 

O^l^D 

and with the change of variables (3.7) and the formula (2.8) we find that 

(3.10) f dP dPo = r *(«, l)l dl = irAiAi (f -i{Ai A- .I 2 ) d* + i d*. 

J Jo o 2 

o^l^d 


Collecting (3.8), (3.9), (3.10) and taking into account (3.5) we have 
^ HR, l)l dl 


+ (1 - {R" ~ rAtAi d" + UAi + A,)cl^- HI, 


(3.11) 
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where p — ab, 72 — A 1 A 2 , R' = (Ai + 25) {A 2 + 25), p(p, 1) is given by (2.4) and 
1) by (2.8). 

For the variance of X and of 7, we have by (3.1) and (3.2) 

= E{X^) - B*(X) = E{Y^ - E\Y) 

/ 

• {ft’ - irAiAi(e + 


which completes the solution of our first problem stated in the introduction. 

4. Second problem. In order to solve the second problem (ii) stated in the 
introduction we will follow the same method of the preceding section. 

Let X be the intersection of the set theoretical sum of the N n-dimensional 
spheres Sn ,r of radius r with the n-interval R, Let us call Y the set of those points 
of R that do not belong to X, that is, 


(4.1) 


X + Y == R. 


The general method of Robbins gives immediately 

(4.2) E{Y) = ft (1 - , E{X) = ft |l = - ‘Vyj 

n n 

where Zf ~ JJ .4,, /^' = U (.4* -f- 25), and Sn,r is given by (2.9). 

1 I 

We now proceed to calculate E{Y“), For this purpose lctQi(i/} ,y\ ^ • • • ,2/n) 
and ( 32 ( 2 / 1 , 2 / 2 , • • • , 2/n) be two points which belong to R and Px{xl, X 2 , • • • ,Xn) 
be the centers of the N s})heres Sn.r • Let us put 

(4.3) d(3. = dyldyl • • • , (Z = 1,2), dl\ = dx{dxl • • • , (z = 1,2, • • •, 


Consider the integi-al 


f dQidQ2dPIdP2 ••• dP^ 
(4.4) J = \ —-- 


extended over the domain defined by 


Qi€ft, Qjeft, ft. «ft', QiP,>r, QiP,>r, U = 1, 2, • • • , iV). 

If wc keep Pi, Pt, Ps, ■ ■ ■ , Pi^ fixed, each point Qi, Qt can vary independently 
over the set Y ; consequently we have 


(4.5) 


j.f 


On the other hand, if we keep Q. and <?2 fixed, the integral of each dft< gives 
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/2' - 2 Sn,r + 1) where fi(Sn,r , 1) is given by (2.12) and I = QiQ^. 

Hence we have 


(4.6) 


J = [ (l _ 

jQl*R,Q%tR \ ti / 

In order to calculate this integral we split it into two parts J = Ji + J 2 , 

n 

according as 0 < Z < 2r or 2r < Z < D, where D = (J!^ A <)*. In the interval 

1 

0 < Z < 2r we introduce the new variables of integration Z, , • • • , 

related to ?/}, 2 / 2 , • • • y\, 2 /i, 2 /L * * * » 2 /n by 


(4.7) y\ = y\ + lii, 

where is given in (2.17). It is found that 


d( 2 /l j 2/2 > * * * > 2 /n » 2/1 » 2/2 » ‘ 1 yn) |n —1 _• n —2 _ 8 


(i = 1, 2, • • • ,n), 

= I Sin <pi sin ^2 • * * sin (pn-2 • 


Hvi,yl, Vn, I, <fii, ••• vn-i) 

Hence we have, 

(4.8) dQidQi = r“‘ dQidadl, 

where d<r denotes the element of area of the n-dimensional sphere of unit radius, 
given by (2.18). The same method used in section 3 gives 

(4.9) Ji = 4.(ff, i)r-‘ dl, 

where 4>(/2, Z) is given by (2.23). 

In the interval 2r < I < D fi{Sn,r , Z) = 0 and we have 


(4.10) 




<f dQidQi — f dQidQif. 

(JOSISD J0SlS2r ) 


Now we have 
(4.11) 


i 




dQi dQ2 = 


and with the change of variables (4.7) we readily find that 
(4.12) • f dQi dQi = r 4>(J?, 1)1"-^ dl. 

J0:Sl^2r Jo 

Collecting (4.9), (4.10), (4.11), (4.12) and taking into account (4.5) and 
(2.23) we have 
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(4.13) 


= C m, J)r-‘ d 


2*“r’” 


2n(2n — 2) • • • 4.2 


*-l *1. **.*■•. <11-4 


2"+“jB„_tr"+* 

‘(n + fc)(n + ft - 2) ... (n + fc - 2A:) 

n n 

wherefi = ^^.,72' = n(^. + 28);S,.risgivenby(2.9),£„by(2.20),M(-S,,r, 0 

by (2.12) and 4>(i?, Z) by (2.23). In particular, for n = 2, we obtain the value 
given by Robbins [3, (30)], by use of (2.8), (2.15) and the equations S 2 ,n = ttt^, 
Ei = 2. For n = 3, the case of ordinary space it follows from (2.16), (2.24) 
and the equations Sz,r = 4 rr, Fa = 4 tt, F 2 = 2 x, that 


E{Y‘ 




IGrr^ + 127rr^Z — ttZ' 


12/e' 


(aitR -t- 2,r(AiAj + .4i4, 


+ 


(4.14) 


AiAz)! + g (Ai + + . 43 ) 2 '^ 2* (22 + 


QO 

-^ ttRt^ 8ir(AiA3 4" A 2 Aa + AaAa)/*^ 

o 


(Ai + A 2 + A8)r "I* ^ r • 


In this case the exact evaluation is easy if one expands the binomial under the 
sign of the integral and integrates term by term. 

From (4.1) we see that 4 = E{X^) - E\X) = E{Y^) - E\Y), Thus, 
from (4.2) and (4.13) we obtain immediately the second moment E{X^) and the 
variance (r\ of X. 


6 . Remark. In the second problem we can substitute the n-intervals R and 
R' by concentric n-dimensional spheres. The problem may then be stated as 
follows: 

Let Sn,a denote a fixed n-dimensional sphere of radius a and Sn,a-^i the con¬ 
centric n-dimensional sphere of radius 0 + 6 . Sn,a a-nd A^n.a+a shall also denote 
the corresponding volumes. Let a fixed number N of n-dimensional spheres 
with radii r (r < min (a, 6 )) be chosen independently with the probability density 
function for the center of each Sn,r constant and equal to l/>Sn,a-H in 
and zero outside this n-sphere. Let X denote the intersection of the set-theo¬ 
retical sum of the N n-spheres with Sn,a \ we wish to evaluate the first two 
moments of the measure of X. 
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It suffices to observe that in this case we have 


(5.1) 4>(S„.a , 1) = n{Sn.a , 0^» = 2a Sn^l.aEn ^ sin" 6 dd 

Jo 

where Sn-i,a is the volume of the (n — 1 )-dimensional sphere of radius a and 
a = arc cos {I/2a), 

The same method used in section 4 gives 

(5.2) E(y) - S.. (i -1;-)”, m . &,{) - (i - , 

E{Y^) = r(l- J) y ^ ^ 

(5.3) ^ 

+ (i - |.sV. - , or-' 

where ^(Sn,a , 1) is given by (5.1). 

In particular, for /z = 2, by use of (5.1), (2.15) and the indefinite integrals 
j arc cos {l/2a)l dl = — a/) arc cos {l/2a) — J l{^a/ — f)^ + constant, 

f ?(4a= - i”)* dl = -iK4a’ - 


+ 2a* arc sin (Z/2a) + constant 


we find that 


E(r-) = 2. 1/ (l - ^5^ ... - (2a“ arc cos(l/2a) 
- ^((4o“ - ly^ I dl+(^l- - 2ir (2o*(2r* - o') arc cos Q 


" ' V \ (a + «)V I V ' 

— 3a' r(o' — r')* + to* + 2r(o' — r')’ — o* arc sin (r/a)^j-. 

For n = 3, we have by (5.1) and 2.16) 

E(r) - 4, f (1 - ~ J (W - + A-eVdi 


+ ^1 - + Wr* - |Tr®j. 

From (5.2) and (5.3) with the use of the relation az = E(X^) — £^(X) = 
E{Y^) ~ J5^(y) we obtain immediately the second moment E{X^) and the 
variance ax of X. 
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ON A TEST OF WHETHER ONE OF TWO RANDOM VARIABLES 
IS STOCHASTICALLY LARGER THAN THE OTHER 

By H. B. Mann and D. R. Whitney 
Ohio State University 

1. Summary. Let x and y be two random variables with continuous cumulative 
distribution functions / and g, A statistic U depending on the relative ranks 
of the and y^a is proposed for testing the hypothesis/ = g. Wilcoxon proposed 
an equivalent test in the Biometrics Bulletin^ December, 1945, but gave only a 
few points of the distribution of his statistic. 

Under the hypothesis/ = g the probability of obtaining a given U in a sample 
of n x*8 and m y^s is the solution of a certain recurrence relation involving n 
and m. Using this recurrence relation tables have been computed giving the 
probability of U for samples up to n = m = 8. At this point the distribution is 
almost normal. 

From the recurrence relation explicit expressions for the mean, variance, and 
fourth moment are obtained. The 2rth moment is shown to have a certain 
form which enabled us to prove that the limit distribution is normal if m, n go to 
infinity in any arbitrary manner. 

The test is shown to be consistent with respect to the class of alternatives 
fM > ffM for every x, 

2. Introduction. Let x and y be two random variables having continuous 
cumulative distribution functions / and g respectively. The variable x will be 
called stochastically smaller than y if /(o) > g{a) for every o. We wish to test 
the hypothesis f = g against the alternative that x is stochastically smaller than 
y. Such alternatives are of great importance in testing, for instance, the effect 
of treatments on some measurement. One may think of x as the values of 
certain measurements in the control group and of y as the values of the same 
measurement in a group which received treatment. In a particular instance 
the protective effect against infection by certain bacteria was investigated. 
Two groups of rats were used in the experiment. The first group receiving no 
treatment, the second group receiving the drug. Both groups were then infected 
with supposedly equally diluted cultures of the bacteria under investigation. 
Most of the rats in both groups died, but the time of survival was measured and 
it was desired to test whether the drug had the effect of prolonging the life of the 
rats. It was desired to make inferences from the effect on rats to the effect the 
drug would have on Kumans. Thus, the only relevant alternative to the hy¬ 
pothesis that survival times are not influenced by the drug is that the survival 
time of those rats which received treatment is stochastically larger than that 
of the control group. 
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3. The U test. Let the quantities a;i, • • • , Xn , t/i, • • • , i/m be arranged in 
order. This arrangement is unique with probability 1 if P(x< = yj) = 0 and 
this follows from our assumption of c<mtinuity. Let U count the number of 
times a y precedes an x. li P(JJ < U) = a under the null hypothesis, the 
test will be considered significant on the significance level ail U < V and the 
hypothesis of identical distributions of x and y will be rejected. 

This test was first proposed by Wilcoxon [1]. His statistic T is the sum of the 
ranks of the t/’s in the ordered sequence of x’s and 2 /^s. In general 


U = mn + 


m{m + 1 ) 
2 


- T 


and this gives a simple way of computing C7. Wilcoxon, however, treated only 
the case m = n and in this case he tabulated only 3 points of the distribution of 
T. Since the test seems of great utility it seemed worthwhile to compute the 
variance, the moments and the limit distribution of U and to investigate the 
class of alternatives with respect to which the test is consistent. 

Although this paper is written in terms of U and the probabilities of U are 
tabulated the results can be easily interpreted in terms of T if so desired. 


4. The distribution of U. Consider now ordered sequences of n x’s and 
m 2 /’s. Since it is only the relation between x and y that matters we replace 
each X by a 0 and each i/ by a 1 . Let U count the number of times a 1 precedes a 
0. Let pnfn( U) be the number of sequences of n O's and m Ts in each of which a 1 
precedes a 0 C/ times. By examining a sequence with the last term omitted we 
arrive at the recurrence relation: 

pnm{U) — pn^ifn{U iw) pnm—l{U)j 

where paW) =0 if C/ < 0 and pioiU), poi{U) are zero or one according 
as C/ 5 *^ 0 or 17 = 0 . 

Under the null hypothesis each of the (m + n)l/mln\ sequences of n O’s and 
m Ts is equally likely. Consequently if PnmiU) represents the probability of a 
sequence in which a 1 precedes a 0 C7 times then 

(1) P.. (to - - ") + p—tto. 

Using the recurrence relation (1) the probabilities ) have been tabulated 
for m < n < 8 (see Table I). For m = n = 8 the distribution of 17 — i{nm + 1) 
differs only a negligible amount from the normal distribution. We shall, in the 
following, derive the mean, the variance, and the fourth moment of C7, and 
prove that the limit distribution of U is normal if n and m both approach infinity 
in any arbitrary manner. 

It is obvious that Pnm{U) = PmniU), 

Since the probability of the ith 1 preceding thejtk 0 is we have 

( 2 ) EnmiU) = nm/2. 
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TABLli. 1 

Probability of Obtaining a U not Larger than that Tabulated in Comparing Samples of 

n and m 


n = 3 



1 

2 

2 

0 

.250 

.100 

.050 

1 

.500 

.200 

.100 

2 

.750 

.400 

.200 

3 


.600 

.350 

4 



.500 

5 



.650 


n = 4 


\ 

\ m 
U \ 

\ 

1 

2 

3 

4 

0 

.200 

.067 

.028 

.014 

1 

.400 

.133 

.057 

.029 

2 

.600 

.267 

.114 

.057 

3 


.400 

.200 

.100 

4 


.600 

.314 

.171 

5 



.429 

.243 

6 



.571 

.343 

7 




.443 

S 


1 


.557 


= 5 


■/ 

/ 

1 

2 

3 

4 

5 

0 

.167 

.017 

.018 

.008 

.004 

1 

.333 

.095 

.036 

.016 

.008 

2 

. 5 ( K ) 

.190 

.071 

.032 

.016 

3 

.667 

.286 

.125 

.056 

.028 

4 


.429 

.196 

.095 

.048 

5 


.571 

.286 

.143 

.075 

6 



.393 

.206 

.111 

7 



.500 

.278 

.155 

8 



.607 

.365 

.210 

9 




.452 

.274 

10 




.548 

.345 

11 





.421 

! 12 





.500 

13 





.579 


n == G 


;/ 

1 

2 

3 

4 

5 

6 

0 

.143 

.036 

012 

.005 

.002 

.001 

1 

.286 

.071 

.024 

.010 

.004 

.002 

2 

.428 

.143 

.048 

.019 

.009 

.004 

3 

.571 

.214 

.083 

.033 

.015 

.008 

\ 


.321 

.131 

.057 

.026 

.013 

5 


.429 

.190 

.086 

.041 

.021 

6 


.571 

.274 

.129 

.063 

032 

7 



.357 

.176 

.089 

.047 

8 



.452 

.238 

.123 

.066 

9 



.548 

.305 

.165 

.090 

10 




.381 

.214 

.120 

11 




.457 

.268 

.155 

12 




I .545 

.331 

.197 

13 





.396 

.242 

14 





.465 

.294 

15 





.535 

.350 

16 






.409 

17 






.469 

18 






.531 
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TABLE I (Continued) 


n « 7 


^9 


■ 

■ 

o 

5 

6 

7 

^B[ 


mSm 



.001 

.001 

.000 




■■ 


.003 

.001 

.001 

2 

.375 

.111 

.033 

.012 

.005 

.002 

.001 

3 

.500 

.167 

.058 

.021 

.009 

.004 

.002 

4 

.625 

.250 

.092 

.036 

.015 

.007 

.003 

5 


.333 

.133 

.055 

.024 

.011 

.006 

6 


.444 

.192 

.082 

.037 

.017 

.009 

7 . 


.556 

.258 

.115 

.053 

.026 

.013 

8 



.333 

. I 5 S 

.074 

.037 

.019 

9 



.417 

.206 

.101 

.051 

.027 

10 



.500 

.264 

. 13 * 

069 

.036 

11 



.583 

.321 

.172 

.090 

.049 

12 




.394 

.216 

.117 

.061 

13 




464 

.265 

.147 

0 S 2 

14 




..)38 

.319 

.183 

.104 

15 




■ 

.378 

.223 

.130 

16 

1 




.438 

.267 

.159 

17 


i 

1 


1 .500 

.314 

.191 

18 


1 

1 

i 


1 .562 

.365 ! 

.228 

19 



1 

1 



.418 

.267 

20 



I 

1 


.473 

.310 

21 




1 


.527 

.355 

22 



1 

1 

1 1 

1 

1 1 


.102 

23 



i 


1 

1 


.451 

21 



1 

1 




.500 

25 



j 


- ! 


.549 
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TABLE I {Continued) 
n =* 8 



1 

2 

3 

4 

5 

6 

7 

8 

t 

normal 

0 

.111 

.022 

.006 

.002 

.001 

.000 

.000 

.000 

3.308 

.001 

1 

.222 

.044 

.012 

.004 

.002 

.001 

.000 

.000 

3.203 

.001 

2 

.333 

.089 

.024 

.008 

.003 

.001 

.001 

.000 

3.098 

.001 

3 

.444 

.133 

.042 

.014 

.005 

.002 

.001 

.001 

2.993 

.001 

4 

.556 

.200 

.067 

.024 

.009 

.004 

.002 

.001 

2.888 

.002 

5 


.267 

.097 

.036 

.015 

.006 

.003 

.001 

2.783 

.003 

6 


.356 

.139 

.055 

.023 

.010 

.005 

.002 

2.678 

.004 

7 


.444 

.188 

.077 

.033 

.015 

.007 

.003 

2.573 

.005 

8 


.556 

.248 

.107 

.047 

.021 

.010 

.005 

2.468 

.007 

9 



.315 

.141 

.064 

.030 

.014 

.007 

2.363 

.009 

10 



.387 

.184 

.085 

.041 

.020 

.010 

2.258 

.012 

11 



.461 

.230 

.111 

.054 

.027 

.014 

2.153 

.016 

12 



.539 

.285 

.142 

.071 

.036 

.019 

2.048 

.020 

13 




.341 

.177 

.091 

.047 

.025 

1.943 

.026 

14 




.404 

.217 

.114 

.060 

.032 

1.838 

.033 

15 




.467 

.262 

.141 

.076 

.041 

1.733 

.041 

16 




.533 

.311 

.172 

.095 

.052 

1.628 

.052 

17 





.362 

.207 

.116 

.065 

1.523 

.064 

18 





.416 

.245 

.140 

.080 

1.418 

.078 

19 





.472 

.286 

.168 

.097 

1.313 

.094 

20 





.528 

.331 

.198 

.117 

1.208 

.113 

21 






.377 

.232 

.139 

1.102 

.135 

22 






.426 

.268 

.164 

.998 

.159 

23 






.475 

.306 

.191 

.893 

.185 

24 






.525 

.347 

.221 

.788 

.215 

25 







.389 

.253 

.683 

.247 

26 







.433 

.287 

.578 

.282 

27 







.478 

.323 

.473 

.318 

28 







.522 

.360 

.368 

.356 

29 








.399 

.263 

.396 

30 








.439 

.158 

.437 

31 








.480 

.052 

.481 

32 








.520 
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We now seek an expression for Enmiv^) where u = U ntn/2. After multiply¬ 
ing (1) by (U - nml2y, using 

Enmiu^) = - nm/2fpnm(JJ) 

u 

and expanding: 

(3) Enmiu ) = — En-lm(u^) + -^- Enm-l{u^) + Uml^y 

n + m n + m 

where Enm(u) denotes the expectation of (C/ — nm/2) in sequences with n O’s 
and m l*s. The initial conditions of (3) are seen by direct calculation to be 

(4) Enoiu^) = Eon^iu^) = 0. 

By substitution Enm(u^) = nm{n + + 1)/12 is a solution of the recurrence 

relation (3) and its initial conditions (4). Hence, it follows by mathematical 
induction that 

(5) Enm(u^) = nm{n + m 1)/12. 

The fourth moment is similarly a solution of the recurrence relation 

( 6 ) En^iu*) = E„-Uu') + Enn,-l(.U*) 

n + m n + m 

+ ^ {2n^m + 2nm“ — w? —nm) 

which is obtained from (1) by multiplication by (C/ — nm/2Y and expansion. 
The initial conditions of (6) are found by direct calculation to be 

(7) £„o(w‘) = EoM) = 0. 

It may be verified that 

(8) Enm{u^) = (5M^m + bnm^ — 2n^ — 2rr? + Znm — 2n — 2m) 

satisfies the recurrence relation (6) and its initial conditions (7) and hence (8) 
follows by mathematical induction. 

To investigate the limit distribution of u as n, m become infinite we investigate 
the rth moment. Following the same procedure as in the case of the second and 
fourth moments and using the symmetry of the distribution to find the odd 
moments zero we get the following recurrence relation. 

(9) EUu^) ^ -l-t (2l)i 

n -T m a-o 

For r = 1, 2 it is known that Enmiu^l is a polynomial in n and m of degree 3r 
and that it is divisible by nm{n + m + 1). Assuming that Enmiu^"’), a < r 
is a pol 3 momial in n and m of degree 3a divisible by n,m{n + wi + 1) we will 



56 


H. B. MANN AND D. R. WHITNEY 


show that it is possible to find a polynomial of degree Sr in n and m divisible by 
nm{n + + 1) which satisfies the recurrence relation (9) for Enm(u^^) and 

also its initial conditions, namely, = Eomiu^^) = 0. 

The last condition is trivially satisfied if Enmiu^'") is divisible by nm{n + m + 1). 
Our method here is to actually substitute a polynomial with undetermined 
coefficients into (9) and show that the coefficients can be obtained uniquely. 
Rearranging (9) we obtain 


EUu^) -^ 

n + m n + m 


+ m \2 q;/4« 




Since for X <j we can write Enmiu^) = nm{n + in + l)Pnm* where Pnm* is 
a polynomial in n, m of degree 3X — 3 the above equation reduces to 


(11) -^ -^ E„n,-r(u^) = nmO’;-’ 

n + m n + m 

where (3*n« * is a polynomial in n, m of degree 3r — 3. 

Now let 

3r~3 

Enmiu^"^) = nmin + m + 1) X) aan'm^ 

•.y-o 

»+yi3r—8 

where an = are to be determined. Substitution in (11) yields: 

2Z aij[{n + m + l)n'm’ — (n — l)(n — l)’'rn’ — (m - l)n\m — 1)^] = QVm’ 
».y 

and rearrangement yields: 

(12) S an + E (-l)’-“(n'm“ + 

i+ykar-s 

Consider first the terms of degree 3r — 3. In this case i + j = 3r — 3 and 
a — i will give 

3r-3 

E ai,r-,-i[n*m^-^' + {i + l)(«’'-’-‘'w’' + 

twO 


or 

(13) 3r X aizr~z~in*rri^~^\ 

«-o 

Equating the coefficients of these terms of degree 3r — 3 to the corresponding 
ones in Q*nm* it is possible to calculate the value of a<ar-8-*, = 0, • • • , 3r — 3). 

We assume now that the o,y are known for i j > 3r — 3 — (fc — 1) and 
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we will find the value of a,^ where i + i = 3r — 3 — fc. Consider then the terms 
in (12) of degree 3r — 3 — fc. These terms will occur when 

+ i = 3r — 3, a = ^ — A:; i + y = 3r — 4, a = f — A; + 1 ; • • • ; 

+ i = 3r — 3 — A;, a = f 

All, but the last, contain coefficients which have already been evaluated. The 
last one reduces to 

(3r - 3) ’ E"* 

t^O 


Thus by equating coefficients aar^k-i for i = 0, 1, • 3r — 3 — A: can be 

evaluated in terms of the coefficients atj already known and those in QnliT*. 
This concludes the proof that Enmi'ii'") is a polynomial in n, m of degree 3r and 
is divisible by nm(ri + m + 1)- 

We now investigate the coefficients of the terms of degree 3r. For X = 1, 2 


Enm{u^) = 


5-31 


12>^ 


(nm)^(n + ^ + 1)^ + terms of degree < 3X). 


We assume this to hold for X < r and we will show that it holds for X = r. Sub¬ 
stitution reduces the right side of (10) to 

“(2r - 3) ••• 5-3*l 


|■(^-3) 


12'-i 


+ mn' 


12^1 


rr\m - + w)”-'] 


(n - iy-^mr\n + 

+ (terms of degree < 3r) 


or 

r(2r — 1) 


(2r - 3) 




|(n + my-’ [ 

• [n(n - + mim - + (terms of degree < 3r - 1)| 

which reduces to 

‘ {nmY{n + + (terms of degree < 3r - 1). 

12^ 

Comparison of coefficients with (13) multiplied by nm gives 

, 3 ^ 3 -.- (2r- 1) ••• 5 . 3.1 

mn 2-r Oi8»-3-i^ w = 

i-O 

or 


12^ 


{nmYiri + mY 


Enn.(u^l 


^') = (2r- 1)^— 5-3j_l + m + ir 


(14) 


+ (terms of degree < 3r). 
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We now wish to show that E„miul is at most of degree 2r in n or m. For 
r = 1, 2 this has already been established. Assuming that it is true for lower 
moments the right side of (10), which reduces to nmQ®nm* is at most of degree 
2r — 1 in n. We again compare coefficients in (12). First, for terms of degree 
3r — 3 we have already seen that n has degree at most 2r — 2. For terms 
of degree 3r — 4 we use i + ^' = 3r — 3, a = z — 1 and z + j = 3r — 4, a = i. 
The first case gives rise to no terms in n of degree greater than 2r — 2 so when we 
solve for the coefficients a, 3 r- 4 ~i the coefficients of terms in n of degree greater 
than 2r — 2 must be zero. The process repeats and we find no terms in n or m 
of degree greater than 2r — 2 in the left side of (12). This gives Enm{ti^) at 
most the degree 2r in n or m. 

Now consider the ratio 


I = , 


EnMl 

[EnmWY 

(2r- 1) ••• 5-3-1 


{nmY{n + m + 


[nm{n m -Y 1)/12]'' 


(terms of degree < 3r; in n or m, < 2r) 


[r?m(n + m + 1)/12]' 
(terms of degree < 3r; in n or m, < 2r) 


{nmy{n + m + I)*" 


= (2r - 1) 5.3-1 + 

Hence 

(15) Lim 7 = (2r — 1) • • • 5-3-1 


and by a well known theorem it follows from (15) that the limit distribution is 
normal. 


6. Consistency of the U test. If / and g are the cumulative distribution 
functions of the x’s and 2 /’s then our null hypothesis \^f=g. The alternatives 
admitted are /(a) > g(a) for every a. Let Ea denote the expectation under the 
alternative. 

Defining 

0 if < 2// 

Xij = 

1 if Xi > Vi 

EA{Xii) = P{xi > yj) =: f gdf < i 

J—00 

EA{XxjXik) = P(xi > 2/y t Xi ^ yY) — f g df ^ 

J—aO 

EAiXikXjio) — P(Xi > yky Xj > 2/ife) “ /* (I /) ^9 ^ 


we have 
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We can now write 

EA(Xij) = J X, EA{xifCik) = J “ €i, EAixikXjk) = i 

where X, ci, €2 are positive numbers. 

We have then 

<^A(.Xij) = J X o'AiXijXik) = A — “t“ X — X* 

(TAiXijXki) = 0 for i7^k,j9^l aAixii^fk) = A ” *2 + X — X* 

Now 

(16) Ea(U) = EA(Xij) = nm/2 — \nm 

iti 

and 

(17) (Ta{U) = S a^Aixij) + X aA(xifCik) + S ffAixu^jk) + 2 (TAixijXjti) 
or 

o\{lf) = nm(n + m + 1)/12 

+ nm[—X*(n + m — 1) + (X — €i)(m — 1) + (X — C 2 )(n — 1)]. 
Let the critical region under the null hypothesis consist of those [7’s satisfying 
nm/2 — U > tncr where lim in == L Then 

n“*oo 

P(nm/2 - U > i„<r\A) P(.E^(U) -U> k-^^) where k = 

(^A 

and by Tchebycheff’s inequality, since for large values of n,w A; < O' 

which by (5) and (17) gives 
P(nm/2 — U > tn<T\A) > 1 

nm(n + m+l) ^ ^ _ 1) + (X - *,)(»» - 1) + (X - ej)(n - 1) 

(in V nm(w + m + 1)/12 — Xnm)^ 

> 1 

1 + , + OT - 1) + (X - «i)(m - J) +(X - <j)(n - 1)] 

n ^ "r ^ ____ 

We obtain then that 

Lim P(nm/2 — U > tno \ A) = I 

n,m-*oo 

which is the requirement for consistency. 
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6. Comparison with other tests. Another test which might seem appropriate 
for the comparison of a control group with a group receiving treatment is the 
test introduced by Wald and Wolfowitz [2J. The test by Wald and Wolfowitz is 
consistent with respect to every alternative However in the case considered 
we are only interested in the alternative hypothesis that measurements in the 
group receiving treatment are stochastically larger than in the control group. 
Intuitively, it seems that the test proposed here is more eflficient for detecting the 
particular alternative considered than the test proposed by Wald and Wolfowitz. 
This intuitive feeling was borne out by the results of the test in the particular 
experiment described in the introduction. All in all, 62 experiments were 
conducted using vaiious bacteria in different solutions and various amounts of 
the protective drug. The U Test gave 14 significant results on the 5% level 
and 4 on the 1% level. The test of Wald and Wolfowitz gave 7 significant 
results on the 5% level and 2 on the 1% level. A final decision between the two 
tests can, of course, only be arrived at on the basis of their power functions, 
which present formidable difficulties. 

In comparing the two statistics it was noted that a slight dislocation of a 
value may cause a significant change in the number of nins easier than it can 
cause a significant change in the statistic proposed here. For instance, in the 
sequence XiX 2 X^XAXb)c^y\y 2 yzy 4 yf>y% both statistics would give a probability less than 
.05. If however, the sequence is slightly altered to X\X 2 X^^XiyiXzy%yzy\yhyi , 
P (number of runs < 4) > .05 while P(C/ < 1) = .002. 

After completion of the present paper it came to the authors attention that 
the U test had already been proposed by K. K. Mathen [3]. However Mathen^s 
distribution of U is incorrect and its derivation erroneous, since it assumes 
independence of the random variables x^j as defined in section 5 of the present 
paper, while obviously x^j and Xxi are not independent. 
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ON THE CONVERGENCE OF SEQUENCES OF MOMENT 
GENERATING FUNCTIONS 

By W. Kozakiewicz 


University of Saskatchewan 

1. Summaiy. The purpose of this paper is to give a few theorems con¬ 
cerning the reciprocal relation between the convergence of a sequence of distribu¬ 
tion functions and the convergence of the corresponding sequence of their 
moment generating functions. 

The paper consists of two parts. In the first part the univariate case is 
discussed. The content of this part is closely related to that of a recent paper 
by J. H. Curtiss [1, p. 430-433], but the results are of a somewhat more general 
nature, and the methods of proofs are different and do not make use of the theory 
of a complex variable. The second part deals with the multivariate case which, 
as far as the author knows, has not been treated before with proofs in as com¬ 
plete and rigorous a way. 

In both the univariate and multivariate cases the proofs are based on the well 
known Ilelly selection principle [2, p. 26] for bounded sequences of mono tonic 
functions. 

2. The univariate case. Let X be a random variable and F{x) its distribution 
function. That is, for any real x, F{x) = P{X < x}, where P{X < x] denotes 
the probability of the event X < x. The function 

vit) = E(e‘^) = r” dF(,x), 

V—00 

in which the integral is taken in the Stieltjes-Riemann sense and is assumed to 
converge in some neighborhood of the origin, is called the moment generating 
function of X (or of F{x)). 

Henceforth we use the abbreviations d.f. and m.g.f. for the terms distribution 
function and moment generating function respectively. The variable t will be 
always real. 

Theorem 1. Let {F»(x)) he a sequence of df.'s. Let M{x) for any fixed 
non-negative x he the least upper hound of the sequence {Fn{^x) + 1 — F«(x)). 
If the sequence {Fnix )} converges on an everywhere dense set of points on the x-axis, 
and if there exists a positive number a s^ich that for any fixed t in the interval | ^ | < a 

(1) Um e'"’‘M(x) = 0, 

x-»+oo 


then: 

(a) (here exists a d.f. F{x) such that lim F„(x) = F(x) at each point of continuity of 

n-*oo 

of Fix); 

(b) the m.g.f.’s of Fix) and F„(x), say <pit) and v>.(<) exist for ] < j < o; 

(c) lim <pnit) = <pit) f<yr\t \ < a and uniformly in each interval 11 j < /S < o. 
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To prove (a), it may be noticed that there exists a function F(x ), non-decreasing 
and continuous on the right, such that lim Fn{x) == F{x) at each point of con- 

n-*oo 

continuity of F{x). But F{x) must be a distribution function. Indeed, we 
have for a; > 0 


(2) F{-x) + 1 - F(x) < Mix-). 

Now from (1), putting ^ = 0, we find that Mix) and consequently Mix — ) 
approach zero as x —> + <». This proves that F(-- oo) = 0 and Fi+co) — 1. 
To prove (b), we notice first that the integral 


<Pnit) = 


dFr^ix) 


(n-1,2,...), 


is convergent for | / | < a. This follows immediately from (1) by applying the 
method of integration by parts to the integrals 


r 


e** dFnix) and 


r 


e*' dFnix), 


which for any t in the interval \t\ < a will be seen to be bounded for all values of 

N. By the same argument, the relation lim Mix—)eJ^^'' = 0, |^1 < a, which 

*-♦+00 

can be easily deduced from (1), together with (2) imply that the integral repre¬ 
senting <pit) is convergent for | ^ [ < a. 

Let now jS be a positive number less than a and let y be such that p < y < a. 
Let My be the least upper bound of for x > 0. Using the method of 

integration by parts and applying (1) we have for | < | < /3 

e" dF„(x) = [1 - F„(N)] e"‘ + </’"" “ W1 dx 

< + Myfi -r . 

7 — P 

We could prove easily that the same inequality is true for the integrals 

"" dFnix), dFix), f "" e"' dFix). 

) J ff *^00 • 

Now let c be any positive number. Because of (3), we have 

( 4 ) [ e'^dFnixXe, f e^^dFixXe, 

for a sufficiently great number Nq , and uniformly with respect to n and t, when 
1 ^ 1 < i9. Clearly, No can be so chosen that Fix) is continuous for a: = zh No- 
Then 

f JVo 

e’’‘dF„{x) = / d“dF{x), 

No 

uniformly for | ^ 1 < /S. 
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The relations (4) and (5) prove that (pn{t) —♦ ip{t) as n —> oo, uniformly for 
\t \ < But can be chosen as near to a as we please; thus (c) is proved. 

Theorem 2. Let {FnWl ^ sequence of d.f,*8 and {^n(01 f'he corresponding 
sequence of m,gf.*s. If <pn{t) exists for | ^ | < a, and if there exists a finite valued 
function (p(t) defined for 1 < | < a, such that lim <pn{t) = <p(t) for j ^ 1 < a, then 

(a) lim = Ofor |^| < a; 

*-♦4-00 

(b) Oiere exists a df. F{x) such that lim Fn{x) = F{x) at each point of continuity 

n-»oo 

of Fix) 

(c) the m.gf, of Fix) exists for | ^ [ < a and is identically equal to (pit) in this interval, 

(d) lim (pnit) = (pii) uniformly in each interval \ t\ < p < a. 

n-*oo 

To prove (a), let t be a number in the interval | /1 < a, and let p be chosen so 
that 1^1 < /3 < a. Then, for x > 0, we have 

Fni-X) + 1 ~ Fr^ix) = [ ^dFniu) + f^^dF^iu) 

J-00 j* 

< dFM + e‘‘'‘dFn{u) 

^ ^ —/3) + fpni0)]> 

Consequently 

+ <Pnim. 

n 

and since the sequences {v?n(—0)) and {(pni0)} are convergent, and therefore 
bounded, it follows that approaches zero as x . 

To prove (b) we may notice that by the Helly selection principle we can 
choose a subsequence {F„j^(x)) which is convergent to some non-decreasing 
function Fix) , at each point of continuity of Fix) . Now the Theorem 1 together 
with (a) imply that Fix) is a d.f. and that the limit of the subsequence {v>njfc(0), 
namely ^(0, must be identical, for | < | < a, with the m.g.f. of Fix). By the 
uniqueness property of a m.g.f. we know that Fix) is uniquely determined by 
(pit), and therefore it follows that every convergent subsequence of {F«(x)) 
approaches the same limit Fix) at each point of continuity of Fix). This is, 
however, equivalent to the statement that the sequence {Fn(x)} itself converges 
to Fix) at each point of continuity of F(x). Thus (b) is proved. We see at 
once that (c) and (d) follow immediately from the Theorem 1. 

Theorem 2 is of course similar to the Theorem 3 in the paper of Curtiss [1, 
p. 432], The proof of (a), however, is not contained in his paper. From the 
Theorems 1 and 2 there follows immediately 
Theorem 3. Let { Fn(x) 1 he a sequence of d.f.^s, and let \(pnit) } he the correspond- 
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ing sequence of m,g.f.*8f which are all assumed to exist for | < | < a. The necessary 
and sufficient conditions for the convergence of {^n(0} i^ the interval | ^ | < a, are: 

(a) lim M(x)e**** = 0, \ t \ < a 

n-*+oo 

(b) the sequence [Fn{x) } converges to a d.f, F(x) at each point of continuity of F{x), 
Furthery the m.g.f. of F(x) exists for | ^ | < a and is equal in this interval to the limit 
of the sequence {v>n(0)* 

In his paper Curtiss gives an example of a sequence {T^nWl d.f.^s which 
converges to a d.f. F(x), while the corresponding sequence {^„(01 of m.g.f.’s does 
not converge to the m.g.f. <p{t) of the d.f. F{x), though both = 1» 2, • • •)> 

and ^(0 exist for all t. It may be easily proved by the direct method that in the 
case considered the condition (a) of the Theorem 3 is not satisfied. 

It is perhaps worth while to notice that the condition (a) of the Theorem 3 may 
be expressed also as follows: 

lim log M{x) < — a. 

X-»+00 

3. The multivariate case. For the sake of simplicity we shall consider here 
the bivariate case only. The results obtained in this chapter, can be, however, 
easily extended to the case when d.f.’s and m.g.f.’s are defined in the Euclidean 
space of any finite number of dimensions. 

Let (Xi, X 2 ) be a random vector variable in the two-dimensional Euclidean 
space, and let F{xi , X 2 ) be its d.f. That is, for any real numbers xi and x^ , 

F{xi , X 2 ) = P{Xi < xi.Xi < X 2 ). 

Let 

Fx(x0 =P{Xi<Xi) =F(xi,+ ^), 

F,{x,) = P\X 2 < X2I = F(+ «>, X2); 

then Fi(xi) and ^ 2 (^ 2 ) are called the marginal d.f.’s of Xi and X 2 respectively. 
The m.g.f.^s of the d.f.’s F{xi , X 2 ), Fi(xi) and F 2 {x^ are defined by the equations: 

J—00 •/—oO 

vSi)) = £( 6 ^“') = a = 1 , 2 ), 

J—ao 

in which the integrals are assumed to converge in some neighborhood of the 
origin. It is easy to see that ipi(ti) = <p(ti , 0) and ^(^ 2 ) = ^(0, 4). 

Theorem 4. Let t^ and (p*{ti, ti) he the m,gf,^s of df.^s F{xi, X 2 ) and 
F*(xi, X 2 ) respectively. If (p{ti , fe) and (p*{ti , < 2 ) exist and are equal in some 
neighborhood of the origin j << j < a*-, (z = 1, 2), then F(xi , X 2 ) = F*{xi , X 2 ) 
identically. 

To prove this theorem, let us introduce two random vector variables (Xi, X 2 ) 
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and (Xt , Xt) of which the d.f/s are respectively F and F*. Consider now two 
random variables 

Z = XA + Xjfe, Z* = X .*<1 + xtk , 

where h and t 2 denote two real numbers not both zero. If ip(t) and ^*(0 are 
respectively the m.g.f.’s of Z and Z*, we have 

<p{t) = , tt^), , tt2). 

Consequently (p{t) = (p*{t) provided that j | < a,-, (t = 1 , 2 ). It follows from 
the uniqueness property of the m.g.f. in the univariate case that the d.f.’s of 
Z and Z* must be identical. Now, according to a theorem due to Cramer 
[3, p. 105], if the d.f.’s of Z and Z* coincide for all pairs of values (h , h) such that 
1 4* I ^ 2 1 5 *^ 0 , the d.f .’s and F* must be identical. It may be worth \vhile to 
reproduce here Cramer’s proof. Let = £;(r*^^**^^^***^) and h) = 

characteristic functions of F and F* respectively. 
Then , tQ and , tk) are the characteristic functions of Z and Z* 
respectively. Since Z and Z* have the same d.f.’s, it follows that 4/ittif = 
for all values of t. Putting ^ = 1, we find that yp{ti , , < 2 ) 

if I 1 + I 4 1 0 . For ti = t 2 = 0f ^(0,0) = ^*( 0 ,0) =» 1 . Therefore , tj) — 

identically, and since the characteristic function uniquely determines 
the d.f., it follows that the d.f. F and F* are identical. 

Theorem 5. Let \Fn{xi , 0 : 2 )) be a sequence of d,f.^s. Let Fin{xi) and F 2 n{x 2 ) 
he respectively the marginal d.f,\s determined by , 0 : 2 ). Let 

Mi{x,) = l.u.b. {F<n(-a:<) + 1 -- Fin{x^] 

n 

1 , 2 ). If there exist positive numbers ai and a 2 such that for 

lim = 0 , (i = 1 , 2 ), 

*,-*+00 

and if {F„(a:i, 0 : 2 )) converges on an everywhere dense set on the (xi , Xo) plane^ 
then: 

(a) there exists a d.f. F{xi , X 2 ) such that lim Fnixi , X 2 ) = F{xi , X 2 ) at each point 

n-*oo 

of continuity of F{x \, 0 : 2 ), 

(b) there exist two positive numbers 5i and 62 , 5* < , such that the m.g.f.*s of 

F(xi , X 2 ) and Fnixi , X 2 ), say , 1^ and <pn{ti , fe), exist for | L -1 < 5,-, (t = 1 , 2 ), 

(c) lim ipn{ti , Q = (p{ti , ^ 2 ) for [ | < 6 <, and uniformly in each two-dimensional 

n-+oo 

interval | L -1 < /3.- < 5», {i = 1 , 2 ). 

To prove (a), we notice that there obviously exists a function F(xi, X 2 ), con- 
tinous on the right with respect to each variable, satisfying the relation 

I^F{xi , X 2 ) = F{xi , x") + F{xi , x' 2 ) - F{xi , a4') - F{xi , x^) > 0 
for Xi < Xi , X 2 < X 2 , and such that 


where Xi > 0 , (i = 

I 1 < a.- 
( 6 ) 
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(7) 


lim Fn(xi,Xi) = F(xi,Xi) 


at each point of continuity of Fixi , *j). We shall prove that F(xi , Xj) is a d.f. 
In fact, it is easy to see that we have for X{ > 0, (* = 1,2), 

(8) F(—Xi , -Xi) < F{-Xi , Xi) < Mi{xi—), F(xi, -Xj) < Afj(x 2 -), 

1 — F(xi , xj) < Mi(xi) + Miixi). 

Now, according to (6), lim M<(x<—) = lim Miixt) = 0, (f = 1,2), therefore it 

follows from (8) that F( — qo , — oo ) = F( — oo , 0 : 2 ) = F{xi , — 00 ) = 0 and 
F{+ <»,+«>) == 1, which proves that Fixi , X 2 ) is a d.f. 

To prove (b), let be the m.gi. of the d.f. F,n(ic<), (^ == 1, 2). Let 

Fi{xi) and F 2 (x 2 ) be the marginal d.f.’s determined by F{xi , X 2 ) and let ipi(ti) be 
the m.g.f. of Fi(Xi), (t = 1,2). 

Now let N' > N > 0 and 


B»(N,Ar',<,,fe) = I" C' e"‘“+'’“dF„(x.,Xs) - f” fdF„(x,, x^) 

n N pN * pN * f > N * 

+ l J J 

N* J-N Jlf J-S* J-N 

+ f" fe*“‘^*'‘dF„(xi, X.) = 7, + 7, + J, + 74 . 

J-jff 


Applying the Schwartz inequality to Ii , we find 

s {C C. {C C. 

But 

(10) r I"' c'*‘‘'dF„(x,,x,) < r e’*‘“dF:„(xi), 

•'-Y J-N* Jn 


and similarly 

(11) r f" /'•‘*(1F„(x,,X2) < f’' e^'‘'dFtn{Xi). 


Let € be any positive number and 7 ,* a positive number less than ai, (z = 1 , 2 ). 
It follows from the proof of the Theorem 1 , taking into account ( 6 ), that the 
integrals representing ipiniU) and (z = 1 , 2 ), exist and are uniformly con¬ 
vergent with respect to n and U , when \ ti\ < ji, {i = 1 , 2 ). Consequently 
we have 


(12) f e^<*dF,M) < «, f e*‘“dF4(x4) < *, (i = 1,2), 

uniformly with respect to n and U when 1 < 1 < 7 », (z = 1 , 2 ), provided that N is 
sufficiently large, say N >Nq, Let us take = 7 </ 2 , (z = 1 ,2). The integrals 
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representing and (i = 1, 2), are obviously uniformly bounded for all 
n and when 1^1 < 7 m = 1 > 2), they are all less than some constant C. Con¬ 
sequently taking into account (9), (10), (11), and (12), we find 

I\ < y /(?€, 

uniformly with respect to n and U when 1 1 < jSi, (i = 1, 2), provided that 

N* > N > No • Since the same inequality is true for h , /i and li , we have 

(13) Rn(N,N\ii,t 2 ) <4\/Ce, 

uniformly with respect to n and U , when [ <»•) < , (z = 1, 2), provided N' > 

N > No. Hence the integral representing (pnik , k) is uniformly convergent for 
M* 1 < fit 9 and consequently convergent for \ U \ < a,/2, (i = 1, 2), since fit 
can be chosen as near to ai/2 as we please. 

Similarly, using (12), we could find 

(14) R(,N, N', ti,td < 4v^, 1 1 < , iV' > AT > ATo 

where 

RiN,N',h,t^ == [" dF(a:i,X2) - T x,). 

This proves, in turn, that the integral representing ^(^i, fe) is uniformly con¬ 
vergent for 1 I < fit and convergent for j 1 < a,/2, (t = 1,2). Thus (b) is 
proved with Si = ai/2, (t = 1, 2). 

To prove (c), let iV' —♦ + «> and N == No in (13) and (14). We obtain 
(16) R„(No , + oo, <x, fe) < 4\/ci, RiNo , + », <i, < 2 ) < 
uniformly with respect to n and U when [ j < . 

Clearly, No can be chosen so that Fi{xi) and F 2 {x 2 ) are continuous for xi = 
X 2 = dzNo- Then 

/•Nq nNo r ATo 

(16) lim / / e'‘'‘+'“‘(iFn(x,,X 2 ) = / / e"‘'‘+"*“dF(xi, xj), 

„-»oo •LJVq *^^0 •^ATq 

uniformly for 1 1 < /3<, (t = 1, 2). 

The relations (15) and (16) prove that 

lim (Pnihy k) = (p(ki k)y 

n —*00 

uniformly for \ k] < , (i = 1,2). The ordinary convergence obviously holds 

for \U\ < ai/2, (^ = 1, 2). 

It follows from the above proof, which refers to the bivariate case, that we 
may take 5,- = a</2, (z = 1,2), in (b) and (c). 

The existence of the corresponding numbers S,, Si < ai , (t = 1, 2, • • • , k), 
in the A:-variate case can be easily established by the repeated application of the 
Schwartz inequality. 



68 


W. KOZAKIEWICZ 


Theorem 6. Let ^„(/i, k), ^,nW, , X 2 ), Fin{x,) and Mi{x^, (i = 1, 2), 

have the same meaning as in the Theorem 5. If ipn{k , exist for [ | < a,*, 

{i = 1,2), and if there exists a finite valued function (p{ti , h) defined for | << | < a*, 
such that lim v>n(<i, = ^{k , ^ 2 ), 1 1 < «», 

n-*oo 

then 

(ii) lim M,[x,) = 0 Jor\ti \ <ai, {i = 1, 2), 

(b) there exists a df. F{xi , .T 2 ), such that lim Fn{xi , X 2 ) = F{xi , X 2 ) at each point of 

n-»oo 

cmitinuity of F(xt, * 2 ), 

(c) the tn.g.f. of F{xi, xj) exists for [ <,• | < a,- and is identically equal to ip{ti , fe) for 
1 <. I < «•, (t = 1, 2), 

(d) lim , fe) = <p{h , k) uniformly for 1 << 1 < /3i < aj, (i = 1,2). 

n-*oo 

To prove (a), it is sufficient to notice that = <pn{k , 0) and ^n(^) = 

^fi(0, < 2 ). Consequently we have 

lim ^in(0 = <p{k , 0), lim (p 2 n{t^ = ^(0, ^ 2 ), | ^* | < a*, (i = 1, 2). 

n-»oo n-*oo 

Therefore (a) follows immediately from Theorem 2. 

To prove (b), we may notice that according to the Helly principle of selection 
applied to the sequence {Fn{xi , 0 : 2 )), there exists a subsequence {Fn*(xi , X 2 )}, 
selected from the sequence {F«(xi , 0 : 2 )} which is convergent to some function 
F{x \, x^ continuous on the right and with non-negative second difference. 
But F{x \, 2 : 2 ) must be a d.f, according to the Theorem 5, since the relation (6) is 
satisfied by the sequence {Fn*(a:i, 2 : 2 )}. Moreover, the limit of the sequence 
Wnuik , ^ 2 )}, namely tpijk , ^), when considered in a sufficiently small neighborhood 
of the origin, is the m.g.f. of F{xi , x^. Since the d.f. is uniquely determined by 
its m.g.f., it follows that every convergent subsequence of {Fn{xi , 0 : 2 )} con¬ 
verges to the same limit F{xi , X 2 ) at each point of continuity of F(xi , X 2 ). This 
is, however, the same as to say that the sequence {Fn{xi , X 2 )} itself converges to 
F(xi , X 2 ) at each point of continuity of F{xi , X 2 ). 

To prove (c), we have to show that the m.g.f. of F{xi , X 2 ), say (p*{k , k)j exists 
for 1^1 < a, and is equal to (p{ti , | 1 < a ,*, (i = 1,2). (We have proved that 

(p*(k , ^) = ^(k , ^ 2 ) only for sufficiently small values of | | and | <2 |). The 

existence of <p*(k , k) for | 1 < a,, (i = 1, 2), can be easily established by the 

method used by Curtiss [1, p. 433). Suppose indeed that <p*(k , < 2 ) does not 
exist at some point (ti , < 2 ), where [ ^| < at, (i = 1, 2). That means that we 
can find a positive number N such that 

(17) r r xs) > , fe") 

J-N J-If 

Since lim Fnixi , X 2 ) = F(xi , X 2 ) at all points of continuity of F{xi , X 2 ), and since 
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N can be so chosen that the marginal d.f/s Fi(xi) and Fzixi) are continuous for 
Xi = X 2 = dbNy it follows that 

(18) Um =: fe‘!'‘+‘S*‘dF’(xi, x,). 

n-*oo •LJV J—y J-y J-AT 

The formulas (17) and (18) give lim ipn(^ , & > (piti , 4), which is impossible 

n—*00 

because lim <pn{ti , k) = <p{ti , k) for | <»• 1 < a,-, (i = 1,2). 

n-*eo 

To prove that ip{ti , k) = <p*(k , k) for 1 <,• | < a;, (i = 1, 2), let (^i, k) denote 
a fixed point such that 1 j < «», (t = 1,2). Clearly, (pn(tk ,tk)y(n = 1,2, • • •)> 
and <p*{tti , tk), considered as functions of the variable t, are m.g.f.’s provided that 
I tti I < a,, (t == 1, 2). (See first part of proof of Theorem 4). Now, according 
to Theorem 2, the limit of the sequence {<Pn(tti , tk) 1, namely <p{ttitti I < a», 
(i = 1, 2), is also a m.g.f. Since <p{tti , tt^ — <p*{tti , in a sufficiently small 
interval containing the point ^ = 0, it follows from the uniqueness property of the 
m.g.f. in the univariate case that <p{tt \, tt^ = <p*itt \, tQ identically for | tU | < , 

a = 1, 2). Putting i = 1, we find <p{ti, k) = ^*(^i, k), M*- 1 < «., {i = 1, 2). 
Thus (c) is completely proved. 

To prove (d), it is sufficient to notice that the sequence {^n(^i, fe) 1 is uniformly 
continuous in each two-dimensional interval 1^,1 < jS,- < a,*, {i = 1, 2), (that 
is, for any € > 0, there exists a positive number d = 5(6) such that 

1 (Pnik , ti) — (pn(ti , t2) I < € 
if 

\t'i - t- I < 5, I < 0.-, I ti 1 < Piy{i = 1,2), in - 1,2, ...)). 

Consequently, the sequence {(Pnik , < 2 )} which is convergent for 1 | < ft , must 

be uniformly convergent if | 1 < , (i — 1, 2). 
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A GENERALIZATION OF TSHEBYSHEV^S INEQUALITY TO TWO 

DIMENSIONS 

By Z. W. Birnbaum, J. Raymond, and H. S. Zuckerman 
University of Washington 

1 . Let Xi, Z 2 , • • • , Z„ be independent random variables with expectations 
E{Xi) = €j and variances = c) ior j = 1,2, • • • , n. The question 

may be asked: What is the upper bound for the probability 

that the point (Xi, X 2 , • • • , Xi) does not fall inside of the ellipsoid 

eM - n 

y-i H 

For n = 1 the answer to this question is given by Tshebyshev^s inequality 


J(X - E(x)f ^ ^ a\X 

L “ J ■“ 


which can not be improved without further assumptions. By a trivial generali¬ 
zation of the argument leading to (1.1) one can prove the inequality 




for any integer n. This inequality, however, can be improved for n > 2. In 

particular, for n = 2, the following theorem will be proved: 

Theorem 1.1, Let X and Y he independent random variables, with expectations 

E(X) = Xo, EiY) = Yq and variances ax , a* • Then, for any s > 0, t > 0 
2 2 

such that ^ ^ we have 


(1.4) I(«, 0 


I ii| + i>i 


p «* 


-(S + t) 


52 ' 


’h'i- #<^<7 + T + s •• 


+ — + 
"T ^2 -r 
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For any given cri, ,« > 0, ^ > 0 such that ~ ^ there exist independent random 

8 t 

variables X and Y with the variances oz, a®, swcfe ihat the equality sign is true in 

(1^). 

This theorem is a special case of the more general statement: 

THBOREiM 1.2. Let W, Z be independent random variables such that 


(1.4) P{W < 0) = P(Z < 0) = 0, 

(1.5) E(W) = X, E{Z) = M, 

( 1 . 6 ) \<ti. 


Then, for any < > 0, ice have 


(1.7) 

where 


(1.8) Mif) 


P{W -\-Z>t)< M(t) 

1 if < < X + M 

X + M_X < — (X + m)_ m 
t t ' t~ t-\ 

if X + /t < < < J(X + + -y/x* + 4fi®) 

if ^(X + 2 m + VX^TV) < t. 


For any given X > 0, /x > 0, X < /x, and t > 0, there exist independent variables 
IF, Z siLch that (I 4 ) and (1,6) arc fulfilled and that the equality sign is true in {1,7). 
Theorem 1.1 is obtained from Theorem 1.2 by writing 


TF = 


(X - Xo)^ 


z = 


(Y - Fo)‘^ 


t = 1. 


2. Before proving Theorem 1,2 we shall derive two lemmas. The first of these 
lemmas deals with more than one variable. Since its proof for general m does not 
present any additional difficulties it will be stated and proven for any number 
m > 1 of variables, although in the proof of Theorem 1.2 it will be used only 
for m = 1. 

Lemma 1. Let (7, Fi, F 2 , • • • , independent discrete random variables 
with only non-negative possible valueSj and let U have a probability distribution 
with the possible values 0 < Ui < U 2 ^ ^ Un and the probabilities P{Ui) = r, 

/or z = 1, 2, • • • , n. TFe consider any three possible values Uj , Uk y Ui of U such 
that 

0 <Ui<Uk<Uiy 

with the corresponding probabilities ry yrk,ri. Then, for any t > 0, there exists a 

random variable U' with the same distribution as U except that the probabilities 
Ti yrk,riofUj,UkyUi are replaced by r, , ri , n such (hat 



72 


Z. W. BIRNBAUM, J. RAYMOND, AND H. S. ZUCKERMAN 


(2.1) E(U') = E(U) 

(2.2) one of /j , r* , r/ zert? 

(2.3) P(17' + Fi + * • • + > 0 > P(^/ + Vi + • • • + > 0. 

Proof: let ry, , rj be written 

(2.4) rj = r/ + ajS, ri[ = r* - jS, rj = n + (1 - a)|8. 

For any a, we then have 

r-y + + ri = ry + + n . 


Choosing 

(2.5) a = (f/, - C/,)/(C7z - Ui) 


we obtain the equality 

U/j + U,r, + Uiu = UiTi + UicTk + Ihr, 


so that (2.1) is true for any /8. 
We obviously have 


( 2 . 6 ) 


/ m \ " 

+ > <) = 'Ljiu 



The variable U' has the same possible values t/» as the variable U. Writing 
P{JJ' = Ui) = r'i , for i == 1, 2, • •' , n, we also have 

(2.7) P {u' + ij F. > «) = E uP (E F. > < - u}j . 

From (2.6), (2.7), and (2.4) we obtain 

F (t/' + E F. > <) - P (f/ + E F. > <) 

(2.8) = apP (E F. > « - u}j - fiP (e F. > < - 

+ (1 - >t - V^. 

For a determined by (2.5), the right-hand side of (2.8) is of the form CjS, and 

will be positive if sign /3 = sign C. If sign C is positive, we choose jS *= r* and 

have, from (2.4), ri = 0, and, from (2.8), the inequality (2.3). If sign C is 

( r ’ Ti \ / / 

—— -- I which leads to either r,- = 0 or r* = 

a 1 — a/ 

0, and again to (2.3). In both cases we have kept the probabilities ry, rl, n 
non-negative as they should be. 
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Lemma 2. Let the discrete random variable U have only the two non-negative 
values Ui K Ui y with the corresponding probabilities ri , r^ y and let t be a given 
number such that 

(2.9) E{U) <t<U2 . 

Then there exists a number a > 0 such that the random variable C/' with the possible 
values 

(2.91) U[ = I7i + a 

V 2 = t 

and the corresponding probabilities n , r 2 , has the properties 

(2.92) 0 < C7i' < U 2 

(2.93) E{U^) = E(U). 

Proof: to have (2.91) and (2.93) it is sufficient to choose 

a == ^ 2 (^2 - t) 
ri 

Then (2.92) is also fulfilled since, in view of (2.9), we have 

rr/ Ti Ui + r 2 U 2 — ^ 2 / E{U) — r 2 < ^ — r 2 t . 

Ul = -= - ^- =: I = (72, 

ri ri ri 

and obviously a > 0 and hence U[ > Ui > 0, 

3. Theorem 1 will first be proven under the assumption that W and Z are 
discrete random variables, each with a finite number of non-negative possible 
values. By repeatedly applying Lemma 1 with m = 1 , [7 = TT, Fi = Z, we 
reduce the number of possible values of W which have non-zero probabilities 
to two, and denote those possible values by TTi < W 2 y and their probabilities 
by Pi and p 2 = 1 — Pi • Then, applying Lemma 1 to the case wi = 1 , 17 = Z, 
Pi = TP, we similarly reduce the possible values of Z to the two non-negative 
values Zi < Z 2 , and denote the corresponding probabilities by qi and 32 = 1 — 91 . 
Throughout all these steps the expectations E{W) = X and E{Z) = /i remain 
unchanged, and P{W + Z > 0 is not decreased. 

For t < \ + fly inequality (1.3) is obviously true, and equality is attained for 
W having the only possible value X with probability 1 and Z having the only 
possible value y with probability 1 , 

For the remainder of the proof we assume t > \ + y. We then have 

t'>h-{‘y^h‘\'Zi'> Wi + Zi. 

If W 2 > ty we may replace it by TFa = < according to Lemma 2 . Similarly, if 
Zj > ty we may replace it by Z 2 = t. The probability P{W + Z >() is not 
decreased in this process. We may thus assume, without loss of generality, that 

W 2 ^ U Zi ^ t. 
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The joint distribution of (T7, Z) has now the possible values represented by the 
four points {Wi , Zi), {Wi , Z 2 ), {W 2 , Zi), {W 2 , Z 2 ). The coordinates of these 
four points and their probabilities fulfill the following conditions 

(3.1) 0 <Wi<\<W 2 <t; 0 < Zi< fjL <Z 2 <t 

(3.2) Pi + P2 == $1 + = 1 

(3.3) PiWi + P 2 W 2 = X, QiZi + ^2^2 = M' 

In view of (3.1), the point {Wi , Zi) always lies below the line W + Z = L The 
other points may or may not lie below that line. Accordingly, we distinguish 
the cases listed in Table I. These clearly include all possible cases since (TTa, Z 2 ) 
can not be below the line TF + Z = ^ without all the other points being below 
that line. 

In case V we have P(W + Z > i) = 0 . 

For the discussion of the remaining cases we note the following relationships 
which follow from (3.2) and (3.3). 


TABLE I 


CaM 

Points below line 

W + Z-t 

Points not below lino 

W + Z - t 

I 

m, Zi) 

(Wt.Zi), (Wi, Z,), (Wt, Z,) 

II 

(Wi,Zi), (W.,Zi) 

iWi.Z,), (.W,,z,) 

111 

(TTi. Zi), (IT, ,Z,) 

(.Wt.Zt) 

IV 

(Wi.z.), (m.Zt), (Wt,z,) 

(JT., Z.) 

V 

(^1 , Zi), (ir,, Zi), (Wi, Z,). , Z2) 

none 


Wi-\ 

X - TFi 

Wi-Wi’ 

Wi-Wi’ 

^ Zi — n 


Zi-Zi' 

Zi-Zx’ 


In case I we have 

(3.41) TFi + Zi < t, W2 + Zi> t, TFi + Z 2 > t, W 2 + Z 2 > t, 

P = P{W + Z > 0 = P2gi + PiQa + P2g2 = 1 - piQi 

j _ W2 ^ \ Z2 — M 

TF2 - TFi Za - Zr 

Since P is a decreasing function of Wi and Zi, we replace Wi and Zi by the 
smallest values compatible with (3.41), namely IFi = ^ — Za, Zi = ^ — TFa, 
and obtain 


P < 1 - 


(TF 2 - X)(Z2 - m) 
(TFa + Za - ty 


= ROV2, Za). 
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For fixed has a minimum at 1^2 = Z 2 + 2X — i and no other 

extremum, hence it assumes its maximum at one or both of the end-points of the 
interval for W 2 which, by (3.1) and (3.41), is 

t- Zi<Wt<t. 

In view of (3.1) we also have t — fi < t — Zi, and hence 
P < Max [R(t — Z 2 ), R{ty Z 2 )]. 

We find 


m - M, z,) = 1 - ^ < 1 - ‘ ^ ^ 


and 


Z 2 — n t — ^ t — n 

R(t, Z 2 ) = 1 - (< - - m) ^ 


. zl 

This last expression has a minimum for Z 2 = 2 and no other extremum, hence it 
assumes its maximum at the ends of the interval for Z 2 which, by (3.41) and 
(3.1), is 

t - Wi < Z2<t. 

From (3.1) we also have / — X < ^ — TTi; and thus 

R(t, W,) < Max [R^"{t - X), = Max . 

Finally, we obtain 


< Max r 


\ + fi 


It - fi' t 


_ XmI 
<2 


Each of the values P = -- - -- , --, ^ can be attained in case I, 

t — fJL t — A t V 

as is shown by the probability distributions 

Wi = 0, 11^2 = ^ 


(3.42) 


(3.43) 


Pi = 1 “ 


,-, P2 = .-, 

Wi = X, TFo = t, Zl = 0, Z 2 « e - X, 

A* ^ 


gi = 1 , 0 ; 


Pi = 1, P2 = 0, = 1 — 


t - X’ 


^2 



■Wi 

= 0, 

Wi = t. 

Zj 0, Z 2 ** 

(3.44) 


X 

X 

1 M 


Pi = 1 

?’ 

Pl = p 

= 1 - 1, Q2 


t 


M 

7* 
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In case II we have 

Wi + Zi < <, W2 + Zi < t, 

(3.51) 


P = PiW + Z > t) ^ piq2 + P 232 = 52 = 


Wi + Z2> t, W2+ Z2> t, 

/X — Zl 


Z 2 — Zi 


This is a decreasing function of Zi as well as of Z 2 and hence takes its'maximum 
for the smallest values of Zi and Z 2 compatible with (3.1) and (3.5), that is for 
Zi = 0, Z 2 = / — X. We thus obtain 

P < 


- < - X 

This upper bound can be attained in case II, as may be seen from the distribution 
TVi = X, W 2 = X, Zi = 0, Z 2 — t — X, 


(3.52) 


Pi = h P2 = h qi = I- 


q2 = 


- X 

Case III is symmetrical with case II and leads to the inequality 

p < -L_. 

t — M 

In case IV we have 

TVi + Zi < ^, W2 + Zi < Wi + Z2 i^, W2 + Z2 ^ 


(3.61) 


p = p{W + Z > 0 = P2q2 = 


(X - WMfi - Zx) 


(W2 - Wi)(Z2 ~ Zi) • 

The right hand side is a decreasing function of each of the variables Wi, W 2 , 
Zi, Z 2 , and hence is increased by chosing for these variables the smallest values 
compatible with (3.61), i.c. 

(3.62) TFi = Zi = 0, W 2 + Z 2 =- t 

for which we obtain 

X u 


P < 


- W2t-W2 




t 


Since has a minimum at TF 2 = and no other extremum, it attains its 

largest value at one of the end points of the interval for 1^2 which, by ( 3 . 1 ), 
(3.61) and (3.62), is 


\ <W 2 <t - il. 


This leads to 


P < Max [P'«(X), - /,)! = Max f-Ar, . 

Lc — A t — 
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The upper bounds —^ ^—, respectively, are attained in case IV for the 

l — A t fi 

probability distribution 

Wi = 0, TF 2 = X, Zi = 0, Zi = t — X, 


(3.63) 


and 


p, = 0, Ps = 1, = 1 - , gj = 


Wi — 0, TV 2 — t — fXj Zi = Of Z 2 — fi, 

qi = 0, ^2 = 1. 


Pi = 1 “ .-, P2 = ,- , 

t — fJL t — IX 


From the preceding discussion we conclude that P = P(W + Z > t) always 
fulfills the inequality 


< Max —, 
J — IX 


M X + 

i 


r - 7 ] - 


for ^ > X + M- Since we have assumed X < /u, we have 


t - IX t 


for 


< > X + M, and therefore 

mt) = Max[^-^, -"p* - for < > X + m- 


It is easily verified that 

X + M Xm 


^ X + fx<i<l{\ + 2ix + V\^ + 4^0 


and 


^ for i (X + 2 m + \/x= + 4 m») < < 

SO that we have U(t) = M(0 as defined in (1.8). For given X, m and any t >\ 
+ M, the equality P = fulfilled for the distributions (3.43), (3.62) and 

(3.63), while the equality P = ^ ^ distribution (3.44). 

I t 

This completes the proof of Theorem 1.2 for discrete random variables. If 
W and Z are independent random variables Avith the cumulative probability 
functions P(IV <w) = F{w) andP(Z <z)= G( 2 ), then each of these cumulative 
probability functions can be uniformly approximated by a step function with a 
finite number of steps, that is by the cumulative probability function of a discrete 
random variable with a finite number of possible values. Since for such variables 
Theorem 1.2 is proven, it also is true for the general random variables W and Z. 
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4. An attempt to extend the method used in proving Theorem 1.2 to more 
than two variables leads to arguments of a prohibitive length. It is possible, 
however, to obtain corollaries of Theorems 1.1 and 1.2 which lead to an improve¬ 
ment of inequality (1.2) for n variables. 

Corollary 2.1 Let Xi, X 21 Xn he independent random variables with 
expectations E{Xj) = ej and variances a{X^ = c]. Then, for any tj > 0, 
j = 1 , 2 , ••• ,n, and any m such that 


m 2 
(Ti 


1 tj /—m-fl ij 


we have the inequality 


V,-i tj 


>l)s 


1 if 1 < Z ^ = Si + 2, 

J-1 tj 

V _ V ^ (2i + ^ 2 ) 

\Uti 

if 2i + Ss < f i [Si + 2Sj + V 2 ? + 4Si] 

Z^-Si-S, if h [2i + 2Si + Vxl + 42^J < t. 

This corollary is a special case of the following corollary to Theorem 1.2 
Corollary 2.2. Let Wi, W 2 f^-*yWn he independent random variables 
such that P{Wj < 0) = 0 for j = 1, 2, • • • , n, and let m he any integer such that 


E EiWi) = X, z E(Wi) = M, 

i—I /—w+l 


X < /i. 


Then, for any t > 0, we have 


' (z Wi > <) < 


M(0 


where M(t) is defined by { 1 . 8 ). 

This corollary follows immediately from Theorem 1.2 by writing 

Tr = Ztr,-, z= Z TTy. 

j-l /-m+1 

To obtain Corollary 2.1, one only has to write in Corollary 2.2 


If some additional assumptions are made on the expectations E{W >) or on the 
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variances , the upper bounds in Corollaries 2.1 and 2.2 may be minimized by 
proper choice of m or of the tj. For example, if all the variances are equal 

2 2 


2 2 
(Tl = 0-2 — 


= cr„ = (7 

and n is even, one obtains the inequality 

1 if < na^ 




1 - 


t — na 


~ n, iina^<e< 


j ^ ^ 3 + ‘s/d _ j 


rur 


ncf. In A 


3 + \/5 


tur < 



DISTRIBUTION OF THE SERIAL CORRELATION COEFFICIENT 
IN A CIRCULARLY CORRELATED UNIVERSE 

By R. B. Leipnik 

Cowles Commission for Research in Economics 
1. Summary. It is desired to find an approximate distribution of simple 

. . X1X2 Xt Xi , , , 

form for the statistic f = —-r 2 "~ (r is an estimate of the serial corre- 

"T * • • 

lation coefficient p in a circular universe) in the case that p 0 in the universe. 
Such a distribution is obtained by smoothing the joint characteristic function 
of the numerator and denominator of the expression for f. The first two mo¬ 
ments are calculated; from these f is seen to be a consistent estimate of p. A 
graph of this distribution for sample size T = 20 and various values of p is given. 

In addition, an approximate distribution for p = Xi + • • • + Xr is derived 
which reduces to the exact (x^-) distribution if p = 0. From a formula which 
yields all moments, it is concluded that, at least up to the degree of approxima¬ 
tion attained, p/T is an unbiased and consistent extimate of 


2. Several writers have investigated the temporally homogeneous stochastic 
process defined by 


( 1 ) 


a:, - px,-i = z, , ^ = 1, 2, • * • , T, I p I < 


where the Zt are unobservable disturbances, normally and independently dis¬ 
tributed with mean zero and variance the Xt are observed variates, and the 
‘‘first observation^’ Xq has a normal distribution with mean zero and such a 
variance al that all later observations have the same variance. Thus we have 

2 


( 2 ) 


o 

Ox = 


a 


1 - p2 


and the joint distribution of a sample of T + 1 successive values is 


(3) 


, ^ _ (L-?!)L 

^1 y * ' * j ^t) * CXp 




2 p(xoXi + • • • + Xt~iXt) + (1 + p*)(a;i + • • • + x\. 




Koopmans ([1], formula 96), by smoothing characteristic values, has obtained 
an approximation to the distribution of the serial correlation coefficient r for the 
case p = 0, where 


(4) 


XqXi + ••• + Xt-iXt 
Xq + ••• + x\ 


‘ Cowles Commission Papers, New Series, No. 21. 
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This result is expressed in the form of a definite integral whose evaluation 
has not so far been effected. 

By considering the related circular stochastic process, where Xq is defined to 
be the same observation as Xt , great simplification is obtained. Here the 
joint distribution of xi, X 2 , • • • , Xr is 

/(X., [- 

(5) {(1 + H- +x\) - 2p{xiXt + • • • + XrXi)} J 

1 — 

“ (1 _ p»)T/2 • 

By smoothing characteristic values, Koopmans ([1], formula 92) found a definite 
integral and Dixon ([2], 3.22) an explicit expression for an approximate distribu¬ 
tion of the circular serial correlation coefficient f, for the case p = 0, where 

x!+ -.. + 4 • 

Dixon’s distribution fio(r) has the simple form 

r (I +1) 

(7) - . (1 - 

Rubin [3] proved these results to be equivalent. On the other hand, R. L. 
Anderson [4] obtained the exact distribution of f in the case p = 0. Madow [5] 
extended this result to the case p 0, using a property of sufficient statistics 
also noted by Koopmans ([1], p. 17) in connection with the non-circular problem. 

It would, however, be difficult to find percentile points or moments from 
Madow’s exact distribution. An approximate distribution of f for p 0, 
together with its moments, analogous to Dixon-Koopmans’ for p = 0, should 
therefore be of interest. The purpose of this paper is to obtain such a distribu¬ 
tion from the circular universe (5). The statistic f is shown to be a consistent 
estimate of p within the limits imposed by the approximation. In addition, an 
approximate distribution for p = x? -f • • • -1- Xy in the case p 5 ^ 0 (which 
reduces to the exact chi-squared distribution when p = 0) is derived, together 
with all of its moments. 

3. We begin by asking about an approximate joint distribution of p and $ de¬ 
fined by 


( 8 ) 


P = Xj -}-•••+ Xy 
q = X1X2 -f* • • • -f- XyXi . 
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Defining <l>{u, v) as the expectation of eKp[i{up + vq)], we have 

♦(“.') - ir ■ • • 17 “® [- i " 

- 2 ^j-^^2 + ?!] dXi---dXT. 

On integration, we find 

(10) <^(«, v) - X(p)[i4(w, w)]"* 

where A{u, v) is the determinant of the matrix associated with the quadratic 
form within the curly brackets in (9). -4(w, v) is a circulant; its value as deter¬ 
mined from the circulant formula ([2], p. 123) is 

(11) A(u, r) = n - 2z cos 
where y and z are defined by 

y = ^^ — 2i<T^u 

( 12 ) 

z = ■ ^ 4- i<r%. 

1 — P* 

To get an approximation A(u, v) to A(w, v) we smooth log A{uy v) by Koopmans* 
method. We have 


L3) log Aiu, 

We define A(u, v) through 


, l>) = S log (i 


2 / — 22 cos ■ 


log A(u, «»)=][ log - 2s cos dt 


in which the summation in (13) is replaced by integration. The integral in (14) 
is easily evaluated ([6], p. 65) giving 


I{u,v)=(^±^ 


Incidentally, had we used = xiXl+i + • • • + XtXt+l in place of §i = j in (9), 
we would have obtained the same expression (15) for A{uj v). 

Setting i{Uy v) = \{p)[A{u, v)]”* we may determine X(p) by the requirement 
0(0,0) = 1. A simple calculation yields the result X(p) = (1 — p*)“^^'*\ (Note 

that = 1 — p** is close to 1 for large values of T), Our result for ^(u, v) 
Mp) 

appears as 


0(w, v) « X(p) I 


y/y ^"" 42^^ 
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The approximate joint distribution of p and q may be written as the double 
Fourier integral 

(17) 6 {p, $) = ^ exp [-t(Mp + og)] dudv 

which we evaluate ([7], 576.3, 914.3) by changing integration variables from 
Uj V to py z and integrating out y and z successively. We obtain finally 


exp l-i(up + vq)] 


(y + 


S(p, 9) = I • 


[■ 10 + - 2<>5l] • 


Changing variables from V,q = pf to p, f, we obtain for F(p, f), the approximate 
joint distribution of p and f, the expression 


Hp. f) = 


T [2c\\ - 


'■(2 + 5) 


(1 - fY 


[1 + p* - 2pf} 


W - P^) ' " ’J' 

We could also have derived (19), following Madow, by noting that for p = 0, 
p and f are independently distributed, p having the chi-squared distribution and f 
having approximately the Dixon distribution (7), and that p and f are sufficient 
statistics for the estimation of p and 

4. The approximate marginal distribution Rp{f) of f is obtained by an easy 
integration from (19) 

*,(„ . (1 - ^)”« 


5,(f) = 




2,7*(1 - p=) 


{1 + p* — 2pf) 


(1 - + p' - 2pf)- 


Our notation is consistent since indeed reduces to the Dixon distribution 
for p = 0. &p{f) has a maximum when 


? = w = 2p (¥^ {(1 + pYt - 1) - Vr(r-2)(i-p*)* + (i + p»)*}. 
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A little manipulation shows that 1 > | fnua I > I P I and that fnua = P asymp¬ 
totically. A graph (Fig. 1) of jSp(f) for T = 20, p = 0, .2, .5, .7, .9 is append^ 
from which it is seen that for | p | near 1, the distribution becomes highly con¬ 
centrated about fjoax. • On differentiating Rp{f) with respect to p and eliminating, 
the envelope of the &p{f) is seen to be 



-10 -3-8-7-6-5-4-3-2-10 I 2 3 4 5 6 7 8 9 0 


P 

Fig. 1. Graph of the Distribution of the Serial Correlation Coefficient in a Circular 

Universe, for T =* 20 


6. Before evaluating the moments of Rp(f) we will pause to obtain the ap¬ 
proximate marginal distribution Pp(p) of p, and its moments. We write 


Pfip) = Pip, = 2 ■ 


[2a\l - P*)]' 


' Li 2 /T l\ 

“ol- £ (f^?)]-r<‘ - [; 


If we define I,(z), the Bessel function of order v and purely imaginary 
argument by 
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( 22 ) 


m = E 


n->0 7lir(y -1“ -j- 1) * 

we obtain ([8], p. 79), if p 5 ^ 0 

(23) PM = I .-'V =P [- i , 


and if p = 0 
(24) 


A(P) = 


(2^^ r^-i 


on performing the integration indicated in (21). Po{v) coincides with the 
exact distribution Po(p)- An expression covering all moments of Pp{p) is 
obtained from (16) by setting v = 0, differentiating, and setting u = 0. We have 


(25) 

hence 


0(t4, 0) = X(p) 




- 7’/2 




(26) 


= (-2cr'?(l - pr"'“ 


di/\ . 2 / 




From (26), we readily find 


(27) 

(28) 


m = T<t\ E [I] = a= 
eip^i = iT<Ty + 2 Tc* 

= 22V‘ 



Thus the unbiased character of p/T as an estimate of is reflected in the ap¬ 
proximate distribution, while (28), which shows that lim = 0, indicates 
that consistency is also reflected. 


6. We now calculate the momcntsof Rpif ). Interchanging the order of integra¬ 
tion in the expression for 5[f*] is justified by the uniform convergence, so we have 
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W’] = f* P[p, f] dpj " j[ [/j dp 


T r2«r»(l - p‘)]-'/* f- 


“^[-£>(r^7)]{/r 


f*(l - exp (mf) df ^ dp 


where m is defined by 


cH 1 — p2) * 


Defining G(m) by 


we have ([8], p. 79) 


G{m) = j (1 - exp (mf) df 

r< ^ _JM_ 


Differentiating each side of (32) k times, we find by (31) and (32) 

ifc ^+1 

O I -2\r/2—1/2 


, (?(ot) =» f*(l - exp (mf) 




^ r-^-r/2 


[7n“^'Vr/2(m)I. 


Using the identity ([8], p. 79) 


az 

and changing the integration variable in (29) from p to m, we obtain finally 
(34) W‘\ = I p"’"'* j[ exp ^ dm. 


For A; = 1, we have ([8], p. 386) 


m = 


2 ’ 

1 + 1 


For fc = 2, after some tedious calculation, we find 

«f^]_ . p'TjT + 1) 

^ ‘ T + 2'^ (T + 2)(r + 4) 

si = -±- fl - pT(T-2) 1 

' T + 2L (T + 2)(r + 4)J 


( 36 ) 
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We note that lim £(f) = p and lim 5? = 0, so that at least to the extent of 

T-^oo 

approximation furnished by fip(f), f is a consistent estimate of p. 

The author wishes to express his gratitude to Dr. T. Koopmans, under whose 
kind direction this paper was written. 
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CONCERNING THE EFFECT OF INTRACLASS CORRELATION ON 
CERTAIN SIGNIFICANCE TESTS 

By John E. Walsh 
Princeton University 

1. Sununary. In practical applications it is frequently assumed that the 
values obtained by a sampling process are independently drawn from the same 
normal population. Then confidence intervals and significance tests which were 
derived under the assumption of independence are applied using these values. 
Often the assumption of independence between the values may be at best only 
approximately valid. For some cases, however, it may be permissible to assume 
that the correlation between each two values is the same (intraclass correlation). 
The purpose of this paper is to investigate the effect of this intraclass correlation 
on the confidence coefficients and significance levels of several well known 
confidence intervals and significance tests which were derived under the assump¬ 
tion of independence, and to extend these considerations to the case of two 
sets of values. 

In the first part of the paper the relations given in Table I are used to compute 
tables which show the effect of intraclass correlation on the confidence coefficients 
and significance levels of the confidence intervals and significance tests listed in 
Table II. The second part of the paper consists of the proofs of the relations 
given in Table I. 

2. Introduction. Let the n values Xi, ,Xn represent a single value of a 
normal multivariate population for which each of the n variables has mean y, 
variance and the correlation between each two variables is p. These n 
values will be called a correlated ‘‘sample.’’ The values xi, • • • , a;„ and 
2 / 1 , • • • , t/m are said to represent two correlated “samples” if they have a normal 
multivariate distribution such that the a:’s have mean y, variance <t*, correlation 
p, the 2 /’s have mean p', variance correlation p', and the correlation between 
each X and y is p". This paper shows that several well known quantities which 
have Student t, Xi or Snedecor F distributions when the values form random 
samples still have these same distributions for correlated “samples” if the quanti¬ 
ties are multiplied by suitable constant factors, where it is to be remembered 
that for normal populations a correlated “sample” is a random sample if and 
only if p = 0 and that two correlated “samples” represent two random samples 
if and only if p = p' = p” = 0. The quantities considered and the corresponding 

n m 

factors are listed in Table I, where 2 = ^ xifn and y = ^ ya/m. Several com- 

1 1 

monly used confidence intervals and significance tests based on these quantities 
and derived under the assumption of randomness are considered, and tables are 
computed which show how the confidence coefficients and significance levels of 
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these confidence intervals and significance tests vary if the values are from 
correlated ‘'samples’^ instead of random samples. Table II contains an outline 
of the confidence intervals and significance tests considered. It is found that 
these confidence coefficients and significance levels can change noticeably when a 
correlated ‘^sample** is considered. This is particularly tnie for the Student 
<-test. For example, in one case it is found that if the sample size is 32 and the 
significance level is .05 when p = 0, then the significance level becomes .23 for 
p = .05. This large change in significance level for a small change in p is ex¬ 
plained by the factor given for the Student ^-distribution in Table I. This 
shows that test results which appear to be “significant*' under the assumption of 
randomness are not necessarily “significant" when correlation is present, even 
though the amount of correlation may be small. The effect of correlation on the 


TABLE I 


Quantity 

Distribution For 

Random Sample 

Factor Multiplying 
Statistic for 
Correlated Samples” 

— /x) \/n(n — 1) (f — n) y/ n{n — 1) 

Student t-distribution 

Qn-lit) dt 

a/~ 

y 1 + (» - Dp 



x*-distribution 

1 

1 — p 

^ 

aiS'* m 

- y)* 

1 

Snedecor F-distri- 
bution 

dF 

1 -p' 

1 — p 


X and Snedecor F tests is not as great as for the Student <-test as can be seen from 
the factors given for the % Snedecor F distributions in Table I. 

3. Effect of intraclass correlation. The relations stated in Table I will now 
be used to investigate the effect of intraclass correlation on the confidence co¬ 
efficients and significance levels of several common types of confidence intervals 
and significance tests which were derived under the assumption of random 
samples. The confidence intervals and significance tests considered are listed 
in Table II, whei’e and are defined in Table I. These particular confidence 
intervals and significance tests have the property that if a is the confidence 
coefficient of the confidence interval listed for a given statistic, then 1 — a is 
the significance level of the significance test listed for that statistic, this relation 
holding whether random samples or correlated “samples" are considered. For 
this reason the tables given in this section will be limited to confidence coeffi- 
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cients; the corresponding significance levels can be obtained by using the above 
relation. 

a. Student t-distrihution. If a random sample of size n is drawn from a normal 
population with mean fjt and variance (denoted by N{n, a)), a confidence 
interval for /x with confidence coefficient e is given in Table II. If the n values 
form a correlated ‘‘sample*’, however, it follows from Table I that the cor¬ 
responding confidence interval with coefficient € is 


^ V' »(n'^^l)(l -^p) ^ ^ n(n^- 


(n - l)p 


1)(1 - P) ■ 


TABLE II 


sut- 

istic 

Para* 

meter 

Exam¬ 

ined 

Coa6dence Interval 
(Confidence Coefficient c) 

Significance Test 
(Significance Level — 1 — <) 

Definitions of 

Constants 

t 


n(n — 1) 

1 i - p1 

g US/Vnin - 1) 

f gn^iCt) dl =* , 

J-t, 

X* 



S 1/xl 

f ,/.-l(x*) rfx‘ - « 
j x« 

F 

(r'2 

0 g <rV<r'^ ^ 


f dF = , 

J F, 


The confidence interval given in Table II can be rewritten as 


l)(l - p) 


^ p g X +«, 




m(n - l)(l - p)’ 


where 



_ 1 - P 

1 + (n — l)p 


Hence if p < 0, a > e and the confidence coefficient of the confidence interval 
in Table II is greater than e. This means that the significance level of the 
corresponding significance test listed in Table II would be less than 1 — « so 
that any test result which would be significant for a random sample would also 
be significant for a correlated “sample” for which p < 0. If p > 0, however, 
€> a and the significance level of the test would be greater than 1 — €. Thus a 
test result which would be significant for a random sample need no longer be 
when p > 0. The effect of positive values of p upon the confidence coefficient 
a = at(p, n) of the confidence interval of Table II is given in Table III for the 
cases € = .95 and .99. Confidence intervals with unequal tails can be treated 
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in a similar manner. It is thus seen that the effect of correlation on the con¬ 
fidence coefficient increases with the sample size n, and that even a very small 
amount of correlation can cause a large change in a. For example, for samples 
of size 16 a correlation of p = .05 will change the significance level from .05 to 
.135; for samples of size 32 a correlation of p = .05 will change the significance 
level from .01 to .102, and from .05 to .23. 

Confidence intervals for p — p' are given by Theorem 5 of section 4. It is to be 
observed that if p = p' = p" and cr = cr' the confidence coefficients are inde¬ 
pendent of p and a. If m = n, p = p', <r = <r', p" = 0, however, the confidence 
coefficients of the confidence intervals for p — p' have the values a = atCp, n) 
given in Table III. 


TABLE III 
Values of a<(p, n) 



0 

.05 

.1 

.2 

.3 

.4 

.5 

n 









.99 


.983 

.974 

.961 

.944 

.920 

4 

.95 


.921 



.805 

.744 

8 

.99 


.959 


.853 

.790 


.95 


.865 


.620 



16 

.99 





.600 

.515 

.95 

.865 

.74 


.54 



32 

.99 


.79 

.63 




.95 


.68 





64 

.99 

.79 






128 

.99 

.68 







6. X’distribution. If a random sample of size n is drawn from iV(p, a^), a con¬ 
fidence interval for <r^ with coefficient € is given in Table II. If the n values form 
a correlated “sample’’, it follows from Table I that the corresponding con¬ 
fidence interval with coefficient c is 

0 ^ ^ SVxUl - P). 

The confidence interval in Table II can be rewritten as 

0 g ^ SVxliX - P), 

where 

Xa = x«/(l - p)- 

Hence if p < 0, a > € and the significance level of the significance test given in 
Table II is less than 1 — e. If p > 0, the significance level of the test is greater 
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than 1 — €. The effect of positive values of p upon the confidence coefficient 
a = ax*(Pi confidence interval listed in Table II is given in Table IV 

for € = .95 and .99. Cases in which the lower limit of the confidence interval 
is not zero can be treated in a similar manner. Table IV shows that the con¬ 
fidence coefficient a = ax*(p» decreases with the sample size n for a fixed value 
of p. Although the effect of correlation for the x*-distribution is not as great as 
for the Student ^-distribution, it does cause a noticeable change in a. For 
example, for samples of size 16 the significance level of the test in Table II is 
changed from .05 to .081 if p = .1 and from .05 to .13 if p = .2. For samples of 
size 32 the significance level is changed from .05 t() .10 for p = .1 and from .05 to 
.19 for p = .2. 

c. Sriedecor f-distribution. If two random samples, one of size n (denoted 
by x\s) and the other of size m (denoted by y’s), are drawn from Niny 
and V(p', <r'^) respectively, a confidence interval for with coefficient e 


TABLE IV 
Valuea of ctx^{p, n) 


p 

0 

.1 

! ^ 

3 

4 

5. 



1 

1 




A 

.09 

.988 

.986 

.983 

.979 

.971 


.95 

.941 

.930 

.918 

.000 

.872 

IP* 

.99 

.982 

.966 

.941 

.890 

.790 

iU 

.95 

.919 

.87 

.79 

.67 

.49 

32 

.99 

.975 

.946 

.867 

.715 

.44 


.95 

.90 

.81 

.64 

.38 

.17 


is given in Table IT. If the values form two correlated “samples”, however, 
it follows from Table I that the corresponding confidence interval with coeffici¬ 
ent € is 

-^(i— 

The confidence interval in Table II can be restated as 


0 n-V/r'** '' ~ / p 


where 


= F.(l - p')/(l - p). 

Thus if p = p', a = « and the significance level of the significance test given in 
Table II remains equal to 1 - If (1 — p')/(l — p) < 1, a > « and the 
significance level is less than 1 - If (1 — pO/(l — p) > 1, however, a < * 
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and the significance level is greater than 1 — Values of the confidence 

coefficient a= ar ^ of the confidence interval listed in Table II are 

given in Table V for c = .95 and .99. Cases in which the lower limit of the 
confidence interval is not zero can be treated in a manner similar to that given 
above. Table V indicates that the effect of correlation on the confidence 
coefficient is not as great forn < m as for n> m. For example, if n = 4, m = 32, 


TABLE V 



^ - — = 1.25, the significance level of the significance test given in Table II is 

1 - p 

only changed from .05 to .069, if i- - = 1.5 from .05 to .087. If n = 32, m = 4, 

1 — p 

1 _ 

-L. = 1.25, however, the significance level is changed from .05 to .094, if 

1 - p 

1 — ^ 1 

- - = 1.6 from .05 to .142. Also it is seen that for fixed -- - , the effect of 

1 — p 1 — P 

intraclass correlation increases with both n and m. 
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4. Analysis. This section contains derivations of the relations stated in the 
first three sections. The method used in these derivations is similar to that used 
in one approach to the analysis of variance and consists essentially in expressing 
each variable as the sum of two quantities, one of which is the same for each 
variable and the other of which is different for each variable. 

Let Xi y ^ • ,Xn represent a correlated ‘‘sample*’, that is, have a normal 
multivariate distribution for which 


E(xi) = M, (t = 1, • • • , n) 

( 1 ) - m )^1 = 

E[(xi — ti)(xi — fi)] = pay 5*^ y = 1, * • * , n). 

Write the Xi, (i = 1, • • • , n), in the form 


= 1? + Xf + , 

n 

where J ^ ii/n and ij, {i, • • • , ?n are independently distributed, ij according to 

N(py <rj) and the according to iV^(0, <r|). The values of X, and a] are chosen 

so that the a;* = + X| + satisfy (1). It is easily proved that it is always 

possible to choose X, and a] so that (1) are satisfied. It is to be remembered 

that p ^ — l/(n — 1) for intraclass correlation. From relations (1) and 
= 1 ? + X| + f < it follows that 

(2) - P), (1=1, 


Theorem 1. The quantity "tj 


(xi — xY has a x-distribution with 


- P)^ 

n — 1 degrees of freedom and is distributed independently of x. 

Proof. Since the are independently distributed according to the same 
normal distribution with zero mean, it follows from (2) that 


n 

Ei^i) 


Z ({. - 1)^ = 


- P)^ 


Z (*i - 


has a x^“distribution with n — 1 degrees of freedom and is distributed inde¬ 
pendently of dc = 1 ? + (1 + X)|. 

T„.»»m 2. //s’-T 


Vl + (n — l)p / 
tribution with n — 1 degrees of freedom. 

Proof. It is easily seen from elementary considerations that 


_?! has a Student t-dis- 

— P 


(x - M)Vn 


(tV 1 + (n - l)p 

has the distribution N{0y 1). Theorem 2 is then an immediate consequence of 
Theorem 1. 

Up to this point a single correlated “sample” of size n has been considered. 
The next part of the analysis, however, will be concerned with properties which 
arise from the consideration of two correlated “samples.” 
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Let zi, • • • , Xn, yi, • • ■ ,y„ have a joint normal multivariate distribution 
such that 


(3) 


Eix.) = M, 

Eiy.) = m', 

E[(xi - = V* 

Eliy. - m ')“1 = 

B[(xi — /x)(x/ — /»)] = p<r*, 

E[iy. - y'Kyt - mOI = p'A 
E[(xi - m)(2/. - mO] = p'V.r'. 


(t = 1, • • • , n) 
(a = 1, • • • , m) 


{i ^ j = 1, • • • , n) 
(a 9^ 0 = 1, ,m) 


Write the Xi and j/o in the form 


X,- = 1 } + Xi{ + Xj|^ + it 

(4) 

2 /« = ^?' + Xif + X 2 I' + fl , 

m 

where {' = and 17, 17', , • • • , {n , , • • * , are independently 

1 

distributed, 17 according to A^(m, o-J), 77 ' according to ATCm', 0 » ff according to 
iVCO, a]), and the according to iV(0, (r|^). The quantities Xi, X 2 , Xi , X 2 , 
orj , , <r| , or I* are chosen so that the Xi and ?/« satisfy (3). It is easily verified 

that it is always possible to choose these quantities so that the Xi and T/a con¬ 
structed in this fashion satisfy (3). In addition it follows from (3) and (4) that 


£7(d) = <^^(1 - P) 

- p'). 

in I 

Theorem 3. - jr ^ -\ -a 53 (2/« ”” vY x*- 

a \i — p) 1 a u — p; 1 

distributions with n — 1 and m — 1 degrees of freedom respectively ^ and are dis¬ 
tributed independently of each other and of x and y. 

1 " 

Proof. From Theorem 1 and (5) it follows that -rr^ -\ 53 ^)* 

U “ P; 1 

1 

and - 727 ;- 7 23 ( 2 /a yY have x^-distributions with n — 1 and m — 1 degrees 

— p) 1 

of freedom respectively. That they are distributed independently of each other 
and of both x and 9 follows from (4). 

— p') 23 

Theorem 4. -- is distributed according to the Snedecor 

^0- “ p) 53 (2/a — yf 

1 

F-distribution hn^i , m-~i{F)dF, 

Proof. This follows from Theorem 3. 
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Theorem 5. _ 

[{x - g) - (m - mQ]V n + m - 2 / /S (^< - S ( 2 /« “ 5 )* 

/ y ‘a‘ 2 ('l -‘7r 7*(1 ~ p') 

where 

<7i = — [1 + (n — l)p] H-[1 + (m — l)p'] — 2p"a(r', 

n m 

has a Student distribution with n + m — 2 degrees of freedom. 

Proof. It is easily seen from elementary considerations that ■ [(x — y) — 

O’! 

(p — p')] has the distribution N (0, 1 ). Theorem 5 then follows from Theorem 3. 

The author wishes to express his appreciation to Professor John W. Tukey for 
valuable assistance and advice in the preparation of this paper. 



ON FAMILIES OF ADMISSIBLE TESTS 

By E. L. Lehmann 
University of California^ Berkeley 

1. Suimnary. For each hypothesis H of a certain class of simple hypotheses, a 
family F of tests is determined such that 

(a) given any test w of 11 there exists a test w' belonging to F which has power 
uniformly greater than or equal to that of w. 

(b) no member of F has power uniformly greater than or equal to that of any 
other member of F, 

The effect on F of various assumptions about the set of alternatives are con¬ 
sidered. As an application an optimum property of the known type A\ tests is 
proved, and a result is obtained concerning the most stringent tests of the 
hypotheses considered. 

2. Introduction. In the theory of testing simple hypotheses, if a uniformly 
most powerful test exists, it is the most desirable test to use. If, as is generally 
the case, such a test does not exist, the choice between tests none of which is 
‘‘altogether better’* than all the others, has to be based on information not con¬ 
tained in the general formulation of the testing problem. If no such additional 
information is available, the choice must of necessity be somewhat arbitrary. 

Now although a single uniformly most powerful test exists only in exceptional 
cases, there will always exist a family F of tests such that 

(a) given any test w of the hypotheses H under consideration and of prescribed 
level of significance, there exists a test w' belonging to F which has power 
uniformly greater than or equal to that of w, 

(b) Pt) member of F has power uniformly greater than or equal to that of any 
other member of F. 

The family F is essentially unique. Arbitrariness occurs only since a test region 
is not uniquely determined by its power function. But since two tests with the 
same power function are equivalent for testing purposes, it is from the present 
point of view immaterial which one is included in F. 

With the same restriction F is essentially the family of admissible tests, a 
test w being admissible if there is no test of the same level of significance which 
has power uniformly greater than or equal to but not identically equal to that of 
w. This definition differs only trivially from the one given by Wald [1, p. 15] 
who defines a test w to be non-admissible if there exists a ly' with power every¬ 
where greater than that of w (except at the hypothetical point). 

F naturally depends on the class of alternatives considered. A restriction 
in the class of alternatives may (although it will not necessarily) diminish F. 
The family F may also be decreased by other additional information; For 
instance a probability distribution may be assumed for the set of alternatives, 
and some properties of this distribution may be presupposed. 

97 
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The determination of the family F, (and a description of the power functions 
of the tests in F) might be considered a solution of the testing problem. The 
solution is not unique and hence does not provide a basis for action. This 
reflects the fact that additional information is needed to make possible the 
unique choice of a best test. On the basis of the available information, F repre¬ 
sents the furthest reduction of the problem that seems possible. On the one 
hand, if the choice of test is to be made from the point of view of power, the only 
contestants for “best test” are the members of F. On the other hand, the 
available information does not give preference to any one member of F over any 
other unless additional principles (such as unbiasedness for instance) are 
introduced. 

It is the purpose of the present paper to illustrate the above notions by deter¬ 
mining F for a very simple case. 


3. Determination of the family F. Let the random variable 
E = (Xi,X2,...;^») 
have a probability density function 
( 1 ) V9 

depending on parameter B, Concerning (1) we shall make the assumptions 
under which Neyman [2, 3] has shown the existence of the tjq^e A\ test of the 
hypothesis 




Assumptions: 

(o) Conditions of regularity: 
The integral 


/ 

Jxo 


e — {xi , * * •, flJn) 
de ” dx\ • • • dXn 


extended over any region w in the sample space, admits of two successive deriva¬ 
tives with respect to 6 under the integral sign, i.e. 

(6) A differential equation: 

If 


vtie) = ^ log p,(e) 
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is not identically zero, and there exist functions of d (but independent of e), 
A and By such that 

(6) = A + B<p$ . 

Under these assumptions Neyman has shown 

A. that the probability density function is of the form 

(7) VM = exp [P{d) + ne)‘Q{e) + E{e)] 

where Q is a monotone function with ~ Q(0) 0 (without loss of generality 

dO 

we shall assume Q monotonely increasing) and 

B. that the t 3 rpe Ai test of the hypothesis H exists, and is given by 

(8) T(e) < Cl , Tie) > 
for suitable choice of ci and C 2 . 

In what follows we shall assume that the permissible first kind error in testing 
H is fixed throughout and has the value c. By a test tc of H we shall always 
mean a test of level of significance e, i.e. satisfying 

(9) f p,,(e) de = (. 

Jto 

Let us consider the family of tests 

(10) w(k) : Tie) < k. Tie) > fik) ; k < fik) 


where fik) is determined by (9). It easily follows from (9) that k can take on 
all values from — qo to A:o, say, where A^o is such that 

(11) fiko) = + 00. 


For the family F of tests {w;(A;)), — <» < k < ko we now state 
Theorem 1. All members of F are admissible, and if w is any admissible test 
not in F, there exists a member of F which has power identical with that of w. 

We first prove the 

Lemma. Let /3w denote the powerf unction of a test w. Then if ki < k2 

i^) ^tc(Ar2) (^) if B 'K. do 

( 12 ) 

^w{ki) id) ^ ^u>(A.2) (^) if d ^ do • 


Proof: Let w denote the complement of a region w. Consider the intei-vals 


(13) 


I = wih) • wik 2 ) 
J = wiki) • wik 2 ). 


I lies entirely to the right of J. T^et d > do, Then 
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(14) 


vM 

Peo(e) 


= c(e) exp {!r(6)[Q(^) - Q(^o)l} 


is a strictly increasing function of T since Q is increasing. Therefore there exists a 


constant 

C such that 




(15) 

< c 

P».(e) " 

if 

T(e) 

is in J 


if 

Tie) 

is in 7. 

Since 





(16) 

/ P«,(fi) de 

= 

! 

P».(e) de 



•'w(Ai) 


we have 





(17) 

1 P»,(e) de 


( 

p,Je) de 





and therefore 




(18) 

p,(e) de < C • p,,{e) de 

= 


^ p»,(e) de 

from which it follows that 




(19) 

[ P*(e) de 

< 

/ 

p#(c) de 



Jwiki) 


which is 

the desired result. 





Proof of Theorem 1. The proof consists of several parts. 

I. Let m be any real number, and assume that there exists a value of h such 
that 


( 20 ) 

( 21 ) 


d 

dB 


^w(^o) — € 

M) |«-»o = ^ 


for w = w{l £). Then w{k) has power uniformly greater than or equal to that of 
any other test satisfying (20) and (21). 

For m = 0 this becomes Neyman’s theorem stating that the type A test is 
also of type Ai . The proof of the theorem however is independent of the value of 

(23) 

and hence carries over to arbitrary m. 

II. If there exists any test satisfying (20) and (21) then there exists a number 
k for which w(k) also satisfies (20) and (21). 
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To prove this let us determine, of all tests satisfying (20), the one which 
maximizes 

(24) Ue) I.-., = £ ~ VM k,. de. 

This can be done by means of the lemma of Neyman and Pearson [4, p. 11] 
which gives sufficient conditions for a region subject to restrictions 

(25) ■ j^f,(e)de = ai, (i = l,---,p), 

to maximize an integral 

(26) f g(e) de. 

Jto 

According to this lemma the desired test is of the form 

(25) ^ p,(e) 1^,, > o-p,,(e) 

provided a value of a exists for which this test satisfies (20). (25) is equivalent to 

(26) P'ido) + T{e) • Q'ido) > a from (7) 

or, since Q'{6o) > 0, to 

(27) Tie) > h. 

Thus, if a number h exists such that the test (27) satisfies (20), this test is the one 
maximizing (24). But such a number does exist, namely /(— cc). Therefore 
wi— 00 ) is the desired test. 

Similarly it is easy to show that of all tests satisfying (20), wiko) minimizes (24). 
But 

(28) 4^wCA)(^) l»-«o = [ 

aa J T<k,T^/(.k) ou 

is a continuous function of A), and therefore takes on all intermediate values, 
which establishes II. 

III. From I. and II. we conclude that given any test w there exists a member 
of F which has power uniformly greater than or equal to that of w. For let w 
be any test of H. From the condition of regularity it follows that its power- 
function has a derivative at So . By II. there exists a value of k such that the 
powerfunction of wik) has the same slope at So , and from I. it follows that 
wik) is uniformly more powerful than w. 

But from the lemma we see that none of the tests wik) is uniformly more 
powerful than any other. Hence all members of F are admissible, and the 
theorem is proved. 

From the lemma and Theorem 1 we can conclude for all members of F the 
following optimum property: 
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Corollary 1: Let w he any testy and let wo be any member of F. Then at least 
one of the two statements 


0w(B) < Pu>o(S) for all $< do 

(29) 

^ for all d ^ do 

must hold. 

The lemma and Theorem 1 also give the following result concerning most 
stringent tests, defined by Wald [1, p. 33]. 

Corollary 2: There exists a uniformly most powerful of all most stringent tests. 
It is that unique member wo of F for which 


l.u.b. fl.u.b. 0ui(d) 


- /3»oW] = ljU-b- [1-Ujb- M) - • 


4, The effect on F of assumptions about the alternatives. Let us next consider 
how a restriction in the set of alternatives effects the family F, From the lemma 
it follows that there is no change as long as the set of alternatives contains 
values of 6 both greater and less than Bo . On the other hand, if the alternatives 
are restricted to values of 6 greater than i say, the family F for testing H 

against these alternatives consists of only a single member, the test w{— oo), 

(and similarly for the other onesided case). This follows from 
Theorem 2: Under conditions a, and b. the test w(— oo) i$ uniformly most 
powerful against the alternatives B > Bo y the test w(ko) is uniformly most powerful 
against the alternatives 6 < Bo, 

Proof: Let w be any test. By Theorem 1 there exists a number k such that 
(30) 0„{B) < Pw(k){B) for all B, 

From the lemma it follows that 

^w{k){B) ^ ^wikfyiiB) ii B ^ Bo 

i^w(,k){B) ^ ^■u,{—to){B) if > ^0* 

Combining (30) and (31) we have the desired result. 

(It is also easy to prove Theorem 2 directly from the Neyman-Pearson lemma.) 
In order to illustrate how the assumption of an a priori distribution of B 
together with some information about this distribution affects F, let us consider a 
special case of the class of hypotheses discussed so far. 

Let 

(32) p.(xi,... yXn) = c . c-*"^**~^'* 

so that E = (Xi, X 2 , - • • , Xn) is a sample from a normal distribution with unit 
variance and unknown mean. We want to test the hypothesis 

(33) H: & ^0, 

We shall show that if B has a probability density function g which is symmetric 
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about the origin, then the family F for testing H consists, as might be expected, 
of a single member, the type Ai test. 

Our problem is to find the test w satisfying 

(34) / po(xi y ••• ,x„) - dxn = € 

Ju> 

and which maximizes 

(35) f ff(e) f pt(xi, ••• ,xn)dxi ■■■ dxn-de. 

J—ao J to 

Inverting the order of integration, Avhich is permissible in this case, the Ney- 
man-Pearson lemma shows the desired test to be of the form 

(36) f g(e)pe(xi , ••• yXn) de > a-po(xi , • • • , Xn) 

provided a value of a exists for which (36) satisfies (34). Substituting from 
(32), (36) becomes 

(37) fix) = f dd > a 

J~oo 

where 

(38) .r = - X) • 

Since 

(39) £2f(x) > 0 

the region (37) is cither empty, which would contradict (34), or else can be 
described by inequalities 

(40) X < ai, X > (12 
where 

(41) /(ffi) = /(a>) 

the latter equation becoming, on substitution from (37) 

(42) f" g{e)e ^ ^ 

J-oo 

If g is an even function, (42) is certainly satisfied when ai = — Oa. Our test 
then becomes 

(43) X < - 02 , x> (h 

which for proper choice of 02 satisfies (34) and is the well known type Ai test. 
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6. Concluding remarks. Let us consider once more a probability density 
function satisfying a. and b. We have seen that the family F for testing H 
against the alternatives 6 $o contains an infinity of elements unless we make 
some additional assumptions. On the other hand, if the principle of unbiased¬ 
ness is accepted, F shrinks to a single element: the type Ai test. 

But unbiasedness does not insure power. Thus conceivably some other test 
might be more powerful than the test chosen, ever 3 rsvhere except in a small one 
sided neighbourhood of do . That this is not so is shown by Corollary 1 to 
Theorem 1. This remark illustrates how intuitively appealing principles and a 
knowledge of the family F may be used in conjunction to arrive at a choice of 
a satisfactory test, when not enough information is available to make the choice 
compelling. 

Finally, it should be pointed out that although we restricted our considerations 
to simple hypotheses, the notions developed also apply to composite hypotheses. 
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CONDITIONAL EXPECTATION AND UNBIASED SEQUENTIAL 
ESTIMATION^ 

Bt David Blackwell 
Howard University 

1. Summary. It is shown that E[f(x) E{y | x)] = E(fy) whenever E(fy) 
is finite, and that aE{y [ x) < ay, where E{y | x) denotes the conditional ex¬ 
pectation of 'y with respect to x. These results imply that whenever there is a 
sufficient statistic u and an unbiased estimate t, not a function of u only, for a 
parameter 6, the function E(t | u), w'hich is a function of u only, is an unbiased 
estimate for 6 with a variance smaller than that of ^ A sequential unbiased 
estimate for a parameter is obtained, such that when the sequential test termi¬ 
nates after i observations, the estimate is a function of a sufficient statistic for the 
parameter with respect to these observations. A special case of this estimate is 
that obtained by Girshick, Mostellcr, and Savage [4] for the parameter of a 
])inomial distribution. 

2. Conditional expectation. Denote by x any (not necessarily numerical) 
chance variable and by y any numerical chance variable for w’^hich E{y) is finite. 
There exists a function of x, the conditional expectation of y with respect to x 
[3, pp. 95-101, 5, pp. 41-44) which we denote, as usual, by E{y | x) and which is 
uniquely defined except for events of zero probability, such that whenever /(x) 
is the characteristic function of an event F depending only on x (i.e. / = 1 when 
F occurs and / = 0 when F does not occur), the equation 

(1) EU(x)E(y 1 x)] = Elf{x)y] 

holds. Now if fix) is a simple function, i.e. a finite linear combination of char¬ 
acteristic functions, it is clear from the linearity of expectation that (1) continues 
to hold. Quite generally, we shall prove 

Theorem 1: The equation (1) holds for every Junction fix) for which E\Jix)y] 
is finite. 

To simplify notation, we write Eiz | x) — ExZ for any chance variable z. The 
following corollary to Theorem 1 asserts simply that the operations Eg and 
multiplication by/(x) are commutative. This fact, which is trivially equivalent 
to Theorem 1, has been stated by Kolmogoroff [5, p. 50]. 

Corollary: If E\fix)y] is finite, then Ex\fix)y] = fix)Exy. 

Proof of Corollary: If gix) is a characteristic function, then EigfExV) = 
^iQfy) by Theorem 1. Since Exijy) is unique, the Corollary follows. 

Proof op Theorem 1: Since Theorem 1 holds when fix) is a simple function 
and the product of a simple function and a characteristic function is a simple 
function, the Corollary holds w’hen/(x) is a simple function. 

1 The author is indebted to M. A. Girshick for suggesting the problem which led to this 
paper and for many helpful discussions. 
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Now let /(a:) be any function for which Eijy) is finite. There is a sequence of 
simple functions/n(a:) such that /n(a;) and i/nW | < \f(x) |. For instance 

we may define /n(a:) = m/n when m/n < f(x) < (m + l)/n, 0 < m < n^,f„(x) 
= m/n when (m — l)/n < f{x) < m/n, 0 > m > -*n^,/nW = 0 otherwise. 

We recall the following proposition of Doob [2, p. 296]: 

(2) \E.y\<E.\y\ 

with probability one. Then, using the Corollary (for simple functions) and 

(2) , we have 1 f^E^y \ = i E^if^y) \ < E.\fny \ < E.\fy \ . Also 

(3) E{fM = ECfnZ/). 

Since the two sequences of functionsare bounded in absolute value by 
the summable functions Ex \ fy \ , \ fy \ , Lebesgiie^s theorem [8, p. 29] applied 
to (3) yields (1). 

In section 3 we shall use the fact that if ii is a sufficient statistic for a parameter 
6 and/is any unbiased estimate for 0, then E(f | u) (which, since u is a sufficient 
statistic, is a function of u independent of 6) is an \inbiased estimate for 6. This is 
obvious, since it follows from the definition of conditional expectation that the 
two chance variables / and E{f\u) have the same expected value. The interest¬ 
ing fact is that the estimate F(/ [ u) is always a better estimate for than / in the 
sense of having a smaller variance, unless / is already a fuiKjtion of u only, in 
which case the two estimates/and E(J | u) clearly coincide. This is simply the 
fact that the variance of the regression function of / on ii is not greater than the 
variance of /. In the case of Gaussian variables, ^vhere the regression is linear, 
this fact has been noted by Doob [1, p. 231].^ Our statement is embodied in 

Theorem 2: If y is finite, so is (t^ExV, (I^Exy < (tSj, with equality holding 
only if Exy = y with probability one. 

Proof: Denote by m the common expected value of y and Exy. Suppose for 
the moment that <r^Exy is finite. By the Schwarz inequality E\yExy\ is then 
finite. Then (tSj = E{y - mf = Eliy - Exy) + {Exy - m)f = E{y - Exyf 
+ a^Exy, since E[Exy{Exy — m)] = E\y{Exy — m)] by Theorem 1. Thus a^y 
exceeds a^Exy by E{y — Exy)^, which is positive unless y = Exy, i.e. 2 / is a func¬ 
tion of X. Thus we obtain the usual decomposition: the variance of y is the 
variance of the regression of 2 / on a: plus the variance of y about the regression of 
y on X. 

To show that a Exy is finite, we require the following 

Lemma (Schwarz inequality): If E{f) and E{g^) are finite, then, with 
probability one, 

Elifg) < 

A proof can be constructed on the usual lines by considering the function 
Q{x, X) = Exif + \g)^. There are, however, certain measure-theoretic difficulties 


* For functions of finite variance it is possible to interpret conditional expectation as a 
projection in Hilbert space, when the statement becomes simply the Bessel inequality. 
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in handling simultaneously the conditional expectations of the family of chance 
variables (/ + instead we shall give a simple direct proof based on the 
ordinary Schwarz inequality for integrals. 

We may suppose f > 0, g > 0 with probability one, since, from (2), 

Elm <BZi\f\\g\) 

with probability one. Unless the Lemma holds there are three positive numbers 
o, h, c with a > he for which the event 

{E^g > o‘, < b, E,{g^) < c} = H 

has positive probability. Then denoting by h the characteristic function of H 
and using the Schwarz inequality for integrals, we have 

aP'(//) < = E\h}g) < E(hf^E^hg^) 

= E[hEM')]E[hE,{g^)] < bcI^{H), 

which is impossible. Tliis completes the proof of the Lemma. 

The Lemma, with / = ?y, = 1, yields El(y) < Exi'i/) with probability one, 

which implies the finiteness of a^Exy and hence completes the proof of Theorem 2. 

3. Unbiased sequential estimation. Consider a chance variable z whose 
distribution depends on a parameter 6, If we have an unbiased estimate t{z) 
and a sufficient statistic u(z) (not necessarily a single numerical chance variable) 
for dj then, as mentioned in section 2, v{u) = E{1 \ ti) is an unbiased estimate for 6 
depending only on u.^ We liave shown that the variance of v is never greater 
than that of and we shall see that it is sometimes much smaller (see example II 
at the end of this section). The estimate obtained in this section for the param¬ 
eter of a sequential process is of the v type; its importance lies in the fact that 
in many cases there is an unbiased estimate t (generally poor) which is a function 
of the first observation, and w hich will consequently be an unbiased estimate no 
matter what sequential test procedure is used. 

Let , a: 2 , • • • be a sequence of chance variables w hose joint distribution is 
determined by an unknown point 0 in a parameter space. A sequential sample 
(test) [9] is determined by specifying a sequence of mutually exclusive events 
Si f S 2 , • • • , where Si depends only on Xi, • • • ,Xt and 

(4) 'E PiS,) = 1 for all 6. 

t-1 

The event St is that sampling stops after the ith observation, and (4) ensures that 
sampling stops eventually. Thus if we define the chance variable n = i when Si 
occurs, n is the size of the sample. 

* It was pointed out by the referee that, strictly speaking, u does not have to be sufficient; 
it is necessary only that t>(u) be independent of 6. The author is indebted to the referee for 
many valuable suggestions. 
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Denote by ui, , * • * any sequence of chance variables such that = 

, • • • , is a sufficient statistic for estimating 0 from Xi, • • • , Xt. There 
will of course be many such sequences {w*}, but it often happens that there is 
one which arises in a natural way from the sequential process; if we are sampling 
from a binomial population, for instance, Ui = number of defectives in the first i 
observations is a sufficient statistic. We shall suppose that the sequential test 
satisfies the following condition 

(6) Si = TF.-C(S, + • • • + Si-i),* 

where Wi is an event depending on Ui only. This condition means that when 
the tth observation is taken, the decision to stop at this point depends only on 
the zth sufficient statistic Ui . For the binomial example mentioned above, this 
means that the decision to stop after i observations depends only on the number 
of defectives observed at that stage, and not on the order in which they were 
observed. The Neyman criterion for Ui to be a sufficient statistic [7, 10, p. 135] 
shows that (6) is no restriction whatever for the sequential probability ratio 
test [9] since the ratio in terms of which the test is defined will be a function of 
Ui only. 

Let 4 , ^ 2 , • * • be any sequence of chance variables such that U is a function of 
Xi, • • • , ; define t ^ U when Si occurs. If E{t) = 0, ^ is said to be an unbiased 

estimate for 6 (relative to the particular sequential test US,}). The theory of 
sequential sampling has been formulated primarily for testing h 3 q)otheses; a 
problem which arises naturally and often is the following: After a sequential 
sample has been obtained, is there an unbiased estimate for 0? Since a sample 
of constant aze is a special case of a sequentially selected sample, we cannot 
hope to find unbiased estimates for arbitrary sequential samples unless such 
estimates exist for samples of every constant size. This is equivalent to the 
existence of a function t{xi) for which E{t) = B for all B. Our problem is to 
discover an unbiased estimate for B which, when n = i, is a function of Ui alone. 
Such an estimate has been found by Girshick, Hosteller, and Savage [4] for 
sequential samples from a binomial population. It turns out that whenever 
there is any unbiased estimate at all for a particular sequential test, there is 
also one of the type described. Thus, if there is an unbiased estimate t for 
samples of fixed size iV, there will be an unbiased estimate of the type described 
for every sequential test requiring at least N observations, since t is itself an 
unbiased estimate for such sequential tests. 

Denote by t any unbiased estimate for B relative to a particular sequential 
test Denote by Wi, hi the characteristic functions of the events TF,, 

C{Si + • • • + Si) respectively, and define u = Ux,v = E{ht^iti | Ui)/E(hi^i | u^) 
when n — i. To justify the definition of v we remark that the event \n = i, 
E{ht-.i I w*) =0) has probability zero, since qhi^i < /i,-i with probability one, 
where q is the characteristic function of the event {E{hi^i | w,) > 0), while 


* For any event A , C(A) denotes the event that A does not occur. 
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EiqK^i) = E[qE{h^i j tOl = E[E(h^i j u,)l » Eih^{). 

Since w, is a sufficient statistic for d with respect to a;i, • • • , , t; is a function of 

u and n only, independent of 6, The msin result of this section is 
Theorem 3. v is an unbiased estimate for d. 

Proof: We shall show that v = E(t\ w, n). This not only shows that t; is an 
unbiased estimate for but also interprets t; in a very simple way and, as men¬ 
tioned above, implies that the variance of v does not exceed that of L It must 

be verified that for every event D depending only on n and u, E(dv) = E(jdt)y 

00 

where d is the characteristic function of D. Now D DSt, and DSt, = AS, 

where D{ is an event depending only on u %. It is sufficient, then, to show 
EidtWjii^iv) = E{dxWthi^it)y where d, is the characteristic function of Z),. Now 

EidxiVtht^iv) = EldxWxK^iEihx-iU | Ux)/E{hx^i 1 ii,)], 

using the definition of v. The function in brackets is /i,-i multiplied by a function 
of Ut ; by Theorem 1 its expectation is unaltered if ht^i is replaced by Eihi-^i 11/,). 
Thus the right member of the last equality equals 

E[dx^tE(ht^itx I Ut)] = EidtWtht^iU) = E(dtV)Jit^it). 

We conclude with two examples: 

I. Binomial and Poisson distributions. Suppose Xi , X 2 y ••• are inde¬ 
pendent with identical distributions, either binomial or Poisson, with parameter 
6, Then t — Xi{ = U for all i) is an unbiased estimate for 6, and it is well known 
that Ut — Xi+ • • • + x, is a sufficient statistic for estimating B from Xi, • • • , x,. 
For any sequential test satisfying (6) our unbiased estimate for B will be 

^ E{ht-iXi \Ut ^ u) ^ Ejht-^iXif) 

\ut = u) E{hi^if) 

when n = if Ut — u, where / is the characteristic function of the event Ui = u. 
Then 

0 

k,(u, t) 

j -0 

p _ kiju, t) 

T, kj{u, i) 

;-0 

where k/u, i) denotes the number of possible sequences xi, • • • , x, for which 
n> if xi+ • • • + Xt = Uf and Xi == j. For the binomial case, this is the estimate 
found in [4]. 

II. S.\MPLES OP CONSTANT SIZE. We Consider the special case where a 


for Poisson 


for binomial 
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sample of constant size N is selected, Xi, • •• yXj^ are independent with identical 
distributions, and the density function for Xi has the form 

(7) pix , 6 ) = r(fl)«(e)*'*’9(a:) 

considered by Koopman [6]®. Suppose further that there is an unbiased estimate 
t(xi) for 6. These conditions will be satisfied, for instance, if 6 is the mean of a 
binomial, Poisson, or normal distribution, with w{x) = t{x) = x, Thenuy 
= w(x\) + • • • + w(xn) is a sufficient statistic. Our estimate v becomes simply 
V = E[t{xi) I UsY Now E[t{x^ I tiA-] = * • • = E\i{xs) | Wjv], since Un is a sym¬ 
metric function of , • • • ,Xn ^ which are independent with identical distribu¬ 
tions. Consequently 

so that 

<r’(y) < = ch{xd/N. 


In the special case w(x) = t{x) = x, we have a == 22 estimate is 

j-i 

simply the mean of the N observations , • • • , Xjv . 
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THE DISTRIBUTION OF THE MEAN 

By E. L. Welker 
University of Illinois 

1. Stmunary. Both population and sample mean distributions can be repre¬ 
sented or approximated by Pearson curves if the first four moments of the 
population are finite. Using the ciz , S chart of Craig [2] to determine the Pearson 
curve type for the population, an analogous al , I chart is derived for the dis¬ 
tribution of the mean. This defines a one to one transformation of al, 6 into 
al , 5. The properties of this transformation ai*e used to discuss the approach 
to normality of the distribution of the mean as dictated by the central limit 
theorem. This is facilitated by superposing on the a] , B chart the al , 8 charts 
for samples of 2, 5, and 10. 

2. Introduction. For any given distribution function of a population, a 
method is available for finding the distribution function of the mean, when it 
exists, that depends on characteristic functions and the Fourier integral theorem. 
For example, characteristic functions have been used to show that the arithmetic 
means of samples from a normal population is normal, and, with minor restric¬ 
tions on non-normal populations, that it is asymptotically normal. The method 
depends, of course, on a knowledge of the exact population distribution. 
Some authors have discussed the approximation of the distributions of sample 
means in special cases by one of the Pearson curves. It is the purpose of this 
paper to consider the complete range of Pearson curves as populations to be 
sampled, then to give the sampling distributions of the mean as approximated 
by the Pearson system, and to discuss the manner in which the distribution of 
the mean approaches the normal curve as dictated by the central limit theorem. 
Since the choice of a Pearson curve depends only on moment relationships, this 
will include the approximation of the distribution of the mean for any parent 
population as based on its moments. Both an algebraic and a graphic analysis 
will be given. 

3. Semivariant and moment relationships. Denote by ak the /;th order 
moment of the population with zero mean and unit variance. Let X* be the A;th 
order seminvariant of the population. Let d*; and X* be the same parameters 
of the distribution of x, the mean of a random sample of size N drawn from this 
parent population. Using properties of the seminvariants of linear functions of 
vai’iables independent in the probability sense, formulas relating these param¬ 
eters [1] are 

^ - <4n-\ 

a* = [a* + 3(i\r - i)]N-\ 
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4. The Pearson system of curves and the distribution of the mean. The 

determination of the Pearson curve will be made in accordance with the scheme 
discussed by C. C. Craig [2]. In this system the curve type is fixed by the 
moment and the constant 

2of4 — Zaz — 6 

^-a. + 3 • 

% 



The scheme for determining the type of curve is shown graphically in Fig. 1 in 
which the aj , 6 plane is divided into areas in which the Pearson curve types are 
noted. The bounding aj , 5 curves are 

5=-^l, 5 = 0, 6 = f, 4 = 0, 

4 = 45(5 + 2), and (2 + 35)4 = 4(1 + 26)^(2 + 5). 

Let 5 denote the value of the 5 function for the distribution of the mean. Then 

5 2a4 — Zaz — 6 
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In terms of moments of the parent population 


5 = 


[<*4 + 3(Ar - 1)“1 ^ al 

L N _J ^ N 


a, + S(N ~ 1) 


- a 


2ai — 3ac3 — 6 

04 + 3 + 6(iV -”l) ‘ 


We see that 5 = 5 for = 1, and 8 < 5 for AT > 1. Both 8 and as approach 
zero as N approaches infinity. These are the values of the constants for the 
normal function. This result is expected from the central limit theorem. 


5. The al, 8 diagram for varying sample size. For every given population 
with finite moments of orders 1 through 4 there exists a Pearson curve represent¬ 
ing or approximating its distribution. This determines a point in the as , 8 
plane. For a given sample size, N, there corresponds a point in the as , 5 pfane. 
If the point (as , 6) is now plotted on the al , 8 plane, we can determine the type 
of Pearson curve which is needed to approximate the distribution of x. The 
transformation of as , 8 into al , 8 enables us to analyze the relationship between 
population distributions and distributions of x. The transforms of the boundary 
curves in the as , 8 plane ^vill constitute an aj, 6 chart corresponding to the one 
for as , 8 shown in Fig. 1. In studying the approach to normality of the dis¬ 
tribution of a:-, it is illuminating to superimpose this as, 8 chart on the ag, 8 chart. 
In order to do this, it is necessary to make certain algebraic changes in the 
equations. 

First eliminate a 4 from the formula for 8 as follows. From 


8 = 


2a4 — 3a3 — 6 
«4 4- 3 


we find 


oti = 


35 + 3a3 + 6 
2 - 8 ‘ 


Substitute this in the expression for 8. Then 


- _ 2a4 — 3al — 6 __ 5(a3 + 4) 

a4 + 3 + 6(Ar ~ 1) al + 4 + 2(N - i)(2 - 8) * 

This formula, in conjunction with 

al = ATaJ 


enables us to write the transformations of the boundary curves. 


Boundary Curve Transformed Curve 

s “ 1 s — — (ATas + 4) 

iVa| + 4 -f 6(Ar ~ 1)' 

-(Ara5+4) 

2(Nal + 4) + 10(Ar- 1)' 

8 = 0 . 


5=0 
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._ 2m\ + 4) 

« = # ^ ~ 5(i\ro| + 4) + ie(A - 1)' 

c4 = 4S(S + 2) ailATal + 4 + 2&(N - l)f 

= 4 i(d^ + 4)[d(J\ral + 8N-4)+ 2N&1 + 8]. 

(2 + 35)a3 = 4(1 + 25) (2 + 5) [5(16A^ + 3iVa? - 4) + 2Nal + 8] 

[JVai + 4 + 2S(Ar - l)]*J^oS 
= 4[5(2Ara| + lOiV - 2) + ATal + 4][6(A^s“3 + SAT - 4) + 2iVa| + 8]. 


S 



Fig. 2. The aj, S and aj, 5 Charts. iV «=* 2 


Fig. 2 shows the chart for distributions of x for iV = 2 by dashed curves 
superimposed on the chart for the population shown by the solid curves, and 
Fig. 3 consists of the same curves for iST = 5 and N = 10. The intervals on the 
population values are 0 < aj < 12 and — 1 < 5 < .4 in Fig. 2, but only part 
of the al range is shown in Fig. 3. In each case the curves for the distribution 
of X cover the interval for , 8 which corresponds to the entire interval shown 
for the population in Fig. 2. Population curves are identified by capital letters 
and the corresponding curves for the distribution of x by the corresponding lower 
case letters. 
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Before discussing the Pearson curve relationships disclosed by these graphs, 
let us analyze some of the geometric properties of the transformation itself. 
Let N be considered as the parameter defining families of curves in the at , 8 
plane corresponding to az = constant and 8 = constant, the systems oi lines 
parallel to the coordinate axes. The transform of aj = fc is = k/N, a system 
of lines perpendicular to 8 = 0, and approaching aj = 0 with increasing N at 
the rate The line a] = 0 is invariant under the transformation, but it is 

not pointivise invariant. 


S S 



Fig. 3. The dj , 5 and a] , d Charts 


The transform of 8 = C is 

CiNal + 4) _ 5 _ C(al + 4/JV5_ 

^ “ Nal + 4 + 2(i\r - 1)(2 - C) ’ 5^ + [4 + 2{N - 1)(2 - C)]iNr-^; 

Solving for al , this becomes 

.2 4C - 8[4 + 2{N - 1)(2 - C)] 

“• - m - c) 

Except for the straight line 8 = 0, obtained when C = 0, this is a system of 
rectangular hyperbolas with asymptotes 
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aj = + 2(i\r - 1)(2 - and 6 = C, 

We are concerned only with the range al > 0. Hence 
-[4 + 2(N - 1)(2 - C)]ir^ 

must be positive for the asymptote to show on the diagram. Since 1 5 | <2, 
and thus | (7 | <2, the expression in brackets is necessarily positive. Hence 
the vertical asymptote is always outside the range of interest and will not show 
on the diagram. However the horizontal asymptotes, S = C, do appear in all 
cases. The h3rperbolas are concave downward if C > 0 and are concave upward 
if C < 0. 

Lines of the pencil 5 = maj are transformed into the hyperbolas 

5 _ indi l( Nal + 4) 
al — 2mal(N — 1) + 4 

for iV > 1. It is clear that (0, 0) is the only invariant point. Every point on 
6 = mal is transformed into a point closer to the origin, the square of the distance 
from the origin changing from 

(m* + l)a\ to . (m- + 

It is easily verified that the h3q)erbolas are asymptotic to 

5 __ mNal (N - 1)(1 + 2m) , .2 _ -4 

1 - 2m{N - 1) [1 - 2m{N - l)p “* 1 - 2m{N - 1) 

As N approaches infinity, these asymptotes approach 

-2 

5 = — ^ and 0:3 = 0. 

An area in quadrant one (four) in the al , 5 plane is transformed into an area in 
quadrant one (four) in the al , 6 plane. The transformed area is nearer the 
origin. 


6. Types of Pearson curves for distribution of sample means. Examination 
of the graphs in conjunction with the above described properties of the trans¬ 
formation shows the following facts regarding the distribution of means of 
samples drawn from populations identified by al and 6. First consider the 
normal function and the three main Pearson types only. 


Parent Po'palaiion 
Normal 
II 
1 / 

IV 

vu 

VI^ 


Distribution of Sample Means 
Normal 
II 

h and II 
Ic7 ,1/ and II 
IV 

YIl and IV 
VI/ , YIl and IV. 
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The transition types were disregarded completely in the above analysis. It is 
worth noting that, disregarding type X, III is transformed into III, VII into 
VII, IIt 7 into IIl , never into lit;, V into IV, but never into V. Type X is 
transformed into type III, never into X. Others follow a similar pattern. 

These moment relationships on the distribution of the mean are not sufficient 
conditions in general. In special cases they are, for example the normal dis¬ 
tribution and the type III (see [3]). They do represent the best approximation 
curve as specified by the Pearson system. We know that in some cases, for 
example type II (see [3]), the distribution of means is not described by a Pearson 
curve. It is clear, however, that the approach to normality is indicated ana- 
l 3 rtically by the transformation aj, 5 to as, 6 and is shown graphically by the 
al , 5 diagram. Skewness and kurtosis in the parent population are reflected 
in the distribution of the mean in small samples. A symmetric distribution 
of the mean requires a symmetric parent population regardless of sample size, 
but the degree of skewness decreases rapidly with an increasing number in the 
sample. The Pearson curve which approximates the distribution of x from a 
bell-shaped parent population is also bell-shaped. The Pearson curve approxi¬ 
mating the distribution of x for samples of V = 10 (Fig. 3) is bell-shaped for 
any parent population with values of al and 5 within the intervals considered. 
For samples of 5 in the same range the approximating curve is either bell-shaped 
or J-shaped, but it is never U-shaped. For samples of 2, even the U-shaped 
distribution is possible, but only with extreme values of al and 6. The point in 
the al , 8 plane corresponding to the normal curve is the only invariant point in 
the transformation. Hence parent populations with parameters not satisfying 
aj = 5 = 0 cannot yield normal distributions of sample mean?. 
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NOTES 

This section is devoted to brief research and expository articles on methodology 
and other short items, 

ON THE STUDENTIZATION OF SEVERAL VARIANCES 


By B. L. Welch 
University of Leeds, England 


1. Introduction. In a recent paper [1] the author considered the problem 
of eliminating several variances simultaneously from probability statements 
concerning the mean of a normally distributed variable. The general situation 
envisaged was as follows. We supposed that we had an observed quantity y 
which could be assumed to be normally distributed about a population mean 

k 

7? with variance o-J = S XtO-?, where the X< are known positive numbers and the 
1—1 

(t\ unknown population variances. It was supposed further that the data 
provided estimates s] of the based on degrees of freedom, and having the 
sampling distributions 


( 1 ) 



and that these estimates were distributed independently of each other and of y. 
The problem was to make statements about the magnitude of the difference 
y — r\ which would involve explicitly only the observed variances s ]. The 
probability of the truth of the statements was also to be entirely independent 
of the population values <r ]. 

The solution was given implicitly in a formal mathematical expression and a 
general process of developing successive terms in a series expansion was de¬ 
scribed. In the present communication a slightly different way of reaching this 
development is provided. 

2. General method. If the /,• are large enough the ratio 


(2) v = 

can be taken to be normally distributed with mean zero and standard deviation 
unity. This suggests that, when the /, are not necessarily large, we might 
approach the matter by seeking some other function 

(3) » = 9{«1, «2, • • • , aj, y - >j} 

which will still be normally distributed with the same mean and standard 
deviation. We shall see that such a function can be found, although the method 
to be followed leads us first to another expression 
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(4) 


y — ri = hial ,«!,••• , 4 , x) 


which is simply the transposed form of (3). Once we have obtained h we can 
solve out from (4) to obtain x. 

Since the distribution of y is independent of s] we have 


m p{,i,')dy . exp{-} ■ 

Transforming therefore to the new variable x we have for given s» 

v(x 1 s') dx = ~~ cxol-i \ dx 

= j{s\ X, 2Xi(r<j dx (say). 


The um-estricted distribution of x is then obtained by averaging over the joint 
distribution of the s \. In order that x should be a unit normal deyiatc we must 
therefore have 


(7) pW = j ■■■ j j{s\x, 2:x.<r<l II {p(s') rfs'll = e 
«2 


We have to substitute from (1) and (G) into (7) and then choose the function 
h{is^, x) in such a manner that the equation is satisfied whatever may be the 
values of the unknown <t\ . To evaluate the function by the methods of numeri¬ 
cal integi’ation is probably impracticable except perhaps in some simple special 
cases. A series development is, however, quite feasible. 

Symbolically we can write 

(8) j {s^, X, (2X,(r<)} = x, ^X.a';} 


where di denotes differentiation with respect to Wt and subsequent equation to 
. Equation (7) then integrates out to give 


(9) 


II 





j[w, X, wX.a.l 


vVt" 


lx» 


i.e. 


( 10 ) 


ej{w, X, SX,(T^} = e 


(say). 


The operator 0 must be expanded in powers of before it (*.an be intcrureted 
When this is done we find 


( 11 ) 

( 12 ) 


{ 4-2 6 ^3 8-4 > 

^0\U\ , 4*^C7*vt CF%^i I I 
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Our procedure now is to find successive approximations to h(s\ x). It will 
be convenient to denote by x) an expression which equals /i(s*, x) to terms 
of order 1//J • Further let Cr+i(s*, x) be a corrective term which when added on 
to hris^j x) will give a result correct to terms in Then to this order we 

shall have from (6) 


(13) 


X, exp{-J 


+ 


_L_exol-i 5!(2V?£<)\ / ^Cr+i(ig, a!) _ g(ZX.tc<)cN-i(«>, x)\ 

\/2X.<7,. '^1 /\ dx 2X.<r^ / 


remembering that the leading term in h(,w, x) is xy /. 
Hence from (10) we find 


(14) 



J hl(w, x)\ dhriw, x) 

^ ‘2X<a^- / dx 

, 1 -U* / ac,+i(g*, x) 

V2X,a^ \ dx 


(<r ) X") 






i.e. 


^ /^-*** ^)\ I ^ hr{y^j a;)\_1— dhr(w, x) -j,* 

Given K we can therefore proceed directly to Cr+i and hence to hr+i. 

3. Application to give terms in l//<. It will be sufficient illustration of the 
method, if we show here how to obtain hi from ho. We have from (15) 


( 16 ) ^ \ 4 _ /x 4 . v ®^*^Acxd/— ^ (^^»’^<)\ j/(SX<w^») _ 

dxV v2x^/^r^^7r/"^\ 2^)jV^)- 


-b» 


i.e. 

(17) 


^ f-jef Ci((r^,x) \ 

•\/2X«r*i / 


, (2X.V,V/0 jt 




where d now denotes differentiation with respect to u and subsequent equation 
to unity 


i.e. 


(18) 

(19) 

w'hence 


a / Cl(<^^ X) \ _ (2XVV/<) 1 , 0-2 _4S 


1 (2XV«//<) ^ 
4 (2X<(t*)* da; 



(a; + x’) 


} 


( 20 ) 


Ciia, x) 



(1 + X* ) (2XW/<) '| 
4 (2X«r^)’ J ■ 
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Hence to the terms in l//< we have 


( 21 ) 


y ^ n ^ h(8^, x) = X 



, (1 W) 

4 (SX.8!.)^ J * 


Solving this out for x we obtain to the same order 


( 22 ) 



(1 + r) 

4 (SX»s*,)* J ■ 


where v equals {y — ‘i?)/\/SX,4 . To order 1//, we may regard x as a unit normal 
deviate and hence determine the probability level corresponding to the observed 
ratio V, On the other hand if we wish to determine the value of y — ri which 
will lie on a given percentage level the expression ( 21 ) is the appropriate one 
to use. 

4. Further discussion. The present development is of course basically 
equivalent to that given in the previous paper. Indeed if we integrate (10) or 
(15) out with respect to x we arrive immediately at the formulae which were then 
obtained and which were illustrated by calculating terms to order l//<. In 
fact when calculating higher order terms it seems best to do this integration 
before carrying out the operation 0 . The object of the present note is really to 
stress the fact that we are simply finding a function of the observations and of 
2 / — which is distributed as a unit normal deviate, whatever the values the 
true <t\ may chance to possess. 

Finally, the remarks following equation (7) above should be somewhat ampli¬ 
fied. The equation asserts that the distribution of any arbitrary function x, 
defined by (3), is 


(23) m . /■ ;■/ 


vifc.r'’” 


1 h?{s^, a;) \ dh{s^, x) 

2 2Xi(Ti J dx 




where h{s , x) is the function obtained by solving out (3) for 2 / — 17 . On carrying 
out the integrations in (23) we shall in general obtain p(x) as a function of x and 
<t\ . Our argument is that if h be chosen properly the <r< will disappear from 
p(x), and X will appear only in the form of the unit normal probability function. 

To find h[s , x) by a direct process of numerical integration would appear to 
involve in the first instance the choice of a net-work of points for x and s\. 
Suppose the range of x is covered by w* points and the range of by n,- points. 
We may then as an approximation look on our task as that of finding the (n,ir,r?,) 
values of /i(s*, x) corresponding to this network. Since (23) is to be true for all x 
and <T <, we can take in turn n< values of , and then (23) can be replaced by 
(n*ir<nO simultaneous equations (it would be necessary to use some formula 
expressing d/i(s*, x) /dx in terms of values of x) at discrete values of x or 
conceivably this may be avoided if we work with the integrated form). With 
a proper choice of the points for x, , and , we might expect to evaluate the 
series /i(«*, x) to any required degree of accuracy, but clearly as a general process 
to be used over a whole range of values/* this approach would be too laborious. 
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It may indeed be queried whether theoretically, with an indefinitely fine 
network of points, we shall be led to a unique function h{s^, x) with the common 
sense properties, which, from general statistical considerations, we know it 
should have in order to be acceptable. As with integral equations of a simpler 
character, the passage from a discrete network to a continuum may raise prob¬ 
lems, but it is the author’s opinion that the infinite ranges of x and s\ give us the 
freedom which we require in the solution. 

The author, however, prefers to approach the problem from the numerical 
behavior of the series, of which (15) gives the general terms. Here the practical 
issue appears to be to investigate the relation between the magnitude of the last 
term retained and the /,•. The author hopes in a further paper to give some 
results of an investigation of this character and also some tables facilitating the 
calculation of /i(s^, x). 
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PROBABILITY SCHEMES WITH CONTAGION IN SPACE AND TIME* 

By FfiLix Cebnuschi* and Louis Castagnetto 
Harvard University 

1. Summary* In many natural assemblies of elements, the probability of 
an event for a given clement depends not only on the intrinsic nature of that 
particular element, but also on the states of some or all of the rest of the elements 
belonging to the same assembly. On the basis of this general idea of “contagion” 
some urn schemes are developed in this paper in which one has contagious 
influence in space and time. The most interesting result found is that in general 
the points of convergence of the probability of the assembly are given by some 
of the roots of an equation p = f ip) and that some of these roots, between zero 
and one, represent stable states of the assembly, or points of convergence, and 
others represent unstable ones, or points of divergence. The two neighboring 
roots, (if they are single), of a root representing a point of convergence are un¬ 
stable values of the probability. Consequently, under certain conditions, the 
limiting probability may be made to have a finite jump by changing the initial 
probability by an arbitrarily small amount. The concrete cases developed in 
this paper can l)e considerably extended by similar methods by assuming more 
complicated and general assemblies and laws of contagion. 

^ On the suggestion of the referee, some parts of the original paper were deleted and 
some mathematical simplifications were introduced. 

* Research Associate at Harvard Astronomical Observatory and Guggenheim Fellow. 
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2. Introduction. In the known probability schemes of contagion of Eggen- 
berger and Polya [1], Greenwood and Yule [2], Ltiders [3], Neyman [4], Feller [5] 
and others [6], as well as in Markoff chains different ways are considered in 
which the previous results in a definite series of trials may influence the proba¬ 
bilities of the future ones. All of these schemes consider possible influences of 
the results of the different trials along the time axis; and consequently might 
be called schemes of contagion in one dimension and one direction. 

In many natural assemblies of individuals or elements, the probability of an 
event per individual or element depends not only on the intrinsic nature of the 
considered element but also on the states of the rest of the elements belonging 
to the same assembly. 

The purpose of this paper is to develop some simple schemes with urns in 
which there is a contagious influence in space and time and to show some of their 
consequences. The method which we have used to treat certain concrete cases 
could be applied to more complicated assemblies and laws of influence in space 
and time. 

3. Scheme of a closed assembly of urns in two dimensions. Let us consider 

a set of N urns arranged on a closed surface in such a way that each one of them 
is surrounded by m others. Let each urn contain a finite number of black and 
white balls. In this paper the probability associated with an urn will refer to 
the probability of obtaining a white ball if a single ball is drawn at random from 
the urn. We shall assume that the initial probabilities are equal for all of the 
urns and that the following law of influence holds: When, after a collective 
trial, one finds that the ball drawn from a certain arbitrary urn, taken as the 
central one, is white and that the corresponding results of the m surrounding 
urns give I white and s black balls, one multiplies the probability of obtaining a 
white ball out of the central urn by the factor if the ball drawn from the 

central urn w^ere black, without changing the given results of the surrounding 
urns, one multiplies the considered probability by the factor ai, 2 ai,i. Under 
the specified conditions, it is easily seen that the probability of obtaining a white 
ball from a definite urn at the i + 1 trial will be, by considering all the possible 
alternatives: 

Pi+i = m! ^ ■ ■ ■ ■ . ^ [p%PiauiT~%qiai,2Y + p<5i(p<a2,i)’*“^(g<02.2)-'i 

( 1 ) 

= fiPi) = + Piqi(piOt2,l + 

where; 


Pi + = 1. 

Consequently pi either converges to a root of the equation p = /(p) or tends to 
inflnity. As a probability greater than one or smaller than zero has no meaning, 
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we have to study the function y = /(p) between zero and one. In (1) we have 
given an implicit form for y = /(p), corresponding to a particular case of influ¬ 
ence; by changing the law of influence we change the function/(p). In general 
one can find graphically the roots of equation p = /(p) by plotting y == /(p) and 
y = p and by determining the intersections of these two lines in the range 
0 < p < 1. Later we shall give the values of these roots for some concrete 
examples. From what we have shown it follows that if, for the considered 
assembly of urns and for especially chosen values of the parameters of inter¬ 
connection and initial probabilities, the probability tends to some equilibrium 
value, this must be a root of the equation p = /(p). As we shall see later, the 
roots in the range 0 < p < 1 may represent stable or unstable states of the 
assembly. 

liCt us consider now a general method for finding the explicit form of the 
function/(p) corresponding to laws of influence similar to the one used by Polya. 

Assume that the trial i results in the drawing of I white balls and s black balls 
from the m urns surrounding the central one. Then we add Iwi white and 
shi black balls to the central urn if the result of the central urn was white, and 
Iw 2 white and sh 2 black balls if it was black. It is easy to show that under these 
conditions the probability in the trial t + 1 is related to the probability in 
trial i by the following formula: 

P.+. = Pi f r<i I- tr^(Pi <r‘ + gi ^tI di 

+ (1 - pO f A <r‘(Pi+ qi dt 

where Wi and Ni are the number of white balls and the total number of balls, 
respectively, in the central urn before trial i. Relation (2) permits us to study 
several interesting schemes. It is easy to see that all the possible schemes which 
can be represented by relations of type (2) give only values of the probability 
in the interval zero and one; and consequently we do not need to make the 
restriction in the analysis of the equation p = /(p) that was necessary in the 
previous scheme, represented by equation (1). 

For the case Wi = h = Ci ,W 2 = b 2 = C 2 j we obtain from (2) 


(3) 


p<H = Pi xr" ~ —h (1 — Pi) 


Ni + Tnci 

If Cl = C 2 , (3) gives 

<4) Pi+i = pi. 

If one takes ci = kiNi and C2 = hNi (3) becomes 


Wi + mc2pi 
Ni + mci 


"-TT W = 

and the equation p = /(p) has, in this case, the roots 0 and 1. 
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When wi = b 2 = kiNi and h = W 2 hNi^ one has to replace hid/dh) by 
k{0/dt^ in the second term of ( 2 ); then if we take m = 2 , 


/p\ ^ _ Vi "h” 2 Aji _s_ ^ ( Vi 

(G) Pi+i - 1 L oi. + Qi 


1 + 2ki 

In particular, if /ji = fe = A;, one obtains 


1 2 P*’ + 

2k2 1 4“ fci 4" A;2 


- 3 


Pi + 2A;! 

1 4-2/ci 


;)= 


/(P.). 




and the solutions of the equation p = fip) arc p = i and 1. By considering 
the behavior of y = /(p) one finds that the stable solution is given by the root 
consequently if one starts with any value of 0 < p < 1 the probability tends 
to the limiting value J. If A^i = 0, 0, by simple calculations, one obtains 

from (6) that the solutions of p = /(p), in this case, are zero and one. 

The equation p = /(p), as given by (6), always has the solution 1. In order 
to have the other two roots real, one has to satisfy: 

HI + 2 fe) (2 + &1 + 3 fcs)* > 4(1 + ki + h) 

( 8 ) 

m + fe)* + 2(fci - fe) - 4 ^I, 


A simple and interesting application of relation (2) is for the case of two urns, 
characterized by m = 1 . From ( 2 ) we obtain: 


where 



Vi + ks 

1 4- fe 



Wi = kiNi , hi = k2Ni , W2 — ksNi , 62 = kiNi . 

The equation p = /(p), as given by (9) has the roots 0 and 1 ; and one may fix 
the value of the third root by conveniently choosing the values of the parameters. 

Appl 3 dng ( 2 ) for an arbitrary value of m and integrating by parts, it is seen 
that in general the equation p = /(p) is of degree m 4 - 2 and consequently, by 
choosing appropriate values for the parameters fci, A: 2 , A;*, fci, each of which may 
be between —1 and 00 , one can expect several roots in the range 0 < p < 1 . 
One can easily generalize our relation ( 2 ) for cases in which WifW 2 ,hi, 62 are 
^ven functions of the probability pi . Even in this most general case it is simple 
to see that one would have a recursion formula of the type pt+i = /(p,) and, as 
in the elementary cases which we have considered, the points of equilibrium of 
the closed assembly of urns will be given by those solutions, in the range 0 < p 
< 1 , of the equation p = /(p), where the derivative of y = fip) is negative. 
Consequently the two neighboring roots, if they are single, of a root representing a 
point of convergence are unstable values of the probability. Therefore, under 
cert^ conditions, the limiting probability may take a finite jump if the initial 
probability is changed by an arbitrarily small amount. This is, we think, the 
most important consequence of the contagion schemes that we propose. We 
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consider that many actual cases of contagion could be better understood by 
schemes of the t 3 rpe that we are studying. 

Let us consider now some simple cases of relation (1). If we take 


ai,i = ai ,2 = ofi ai ,2 = a 2 ,i = ^2 and m = 2, 

representing a closed ring of urns, one obtains; 

Pi+i = + Pt^i{0t2Pi + aiQiY 

= p\ + (Pt* — Pi) [{oil + a2f — 4 a?] = f(pi). 

The equation p = /(p), corresponding to this recursion formula, always has the 
solution p = 0. The other two solutions are given by 


( 11 ) 




These roots will be between 0 and 1 when 

2 < «! + a2 

1 > Ofi < a?2 

We would have Pi > 0 and P 2 < 0 if 

2 < ai + a2 
1 < CKl < a2 

and Pi = Pi when 

(14) ai + 0:2 = 2, ai ^ 1. 


( 12 ) 


(13) 


or (120 


or (13') 


2 > ai + 0(2 

1 < ai > 0(2 

2 > ai + ai 
1 > «! > 0(2 , 


Let us now study the general behavior of (10). For the conditions (12') we 
have; 

(15) Pi+i - Pi = a^Pt^Pi - Pi) {pi - P 2 ) 

where a* = 4 a? — (ai + > 0. 


If 0 < Pi < Pi, one obtains from (15) by use of elementary algebra: 


(16) 


Pi+i - Pi I 
Pi — Pi \ 


= 1 (^’z - Pi) 1 



< 1 . 


Consequently if pi > Pi the sequence p< increases monotonically. Otherwise 
p,+i will lie between Pi and pi and will tend to Pi without ever reaching the other 
side of this point. In a similar way it is possible to prove the convergence to a 
constant for the most general equations of the type p = /(p) when they have 
roots between zero and one. 

Let US give some numerical results. For ai = 0.95 and ai = 1.1, from (10) 
one obtains: Pi = 0.1 and Pi = 0.9. It is easily seen that, in this case, if 
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0 < Pi < 0.1, the limiting value of will be zero; if pi > 0.1, the limiting value 
will be 0.9. The interesting point is that if the initial probability is in the 
neighborhood of 0.1, an infinitesimal change in its value may produce a finite 
change in the stable limiting probabilities; and that for the initial probability 
equal 0.1 one would have an unstable equilibrium of the system. This con¬ 
sideration shows why it is important to know how the probability p,- converges 
towards a certain point. As wc have previously shown, the points of con¬ 
vergence are roots of the eq. p = /(p) but there roots which are not points of 
convergence. 

Similar reasoning could be applied to more complicated systems belonging to 
our general scheme of contagion. Consequently, the most important result is 
not that the considered assembly may have a probability tending to some value 
in the range 0 < p < 1, but that under certain conditions the limiting probability 
may jump from one value to another by changing the initial probability by an 
arbitrarily small amount. 
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FITTING CURVES WITH ZERO OR INFINITE END POINTS 

By Edmund Pinnet 
Oregon State College 

The problem of determining a suitable equation to fit an empirically deter¬ 
mined curve over a given interval has been of great importance in statistical 
work, in experimental science, and in engineering technology. Since infinitely 
many types of equations may be made to fit the data with required accuracy, 
the choice of a “suitable^’ type of equation depends on the qualitative nature 
of the empirical curve, on the use to which the equation is to be put, and upon 
considerations of simplicity. 

As a function type, the polynomial has, because of its simplicity, been enor¬ 
mously useful. The function type studied here is a little more general than the 
polynomial type, being particularly useful in the case of empirical curves that 
become zero or infinity at one or both ends of the interval. 

Without loss of generality the interval in which the equation is to fit the curve 
may be taken as 0 < x < 1. It is assumed that, by numerical means or other- 

f' 

wise, a finite set of moment Mm = / dx may be computed, y being the 

Jo 

ordinate of the empirical curve. 
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The problem to be considered here is that of determining a function f(x) of 
the form 

(1) f(x) = *“(1 - x)" EpO,*', R(a) > -1, R(^) > -1 

0 

such that 

(2) f dx = Mm 

Jo 


as m ranges from zero to the number of the highest moment known, fix) is 
then an approximation to y which may be written 

(3) y^fix). 


Theorem 1®. Given a finite set of moments mo , Mi , Ma , • • * , Mn , and given that 
Ria) > —1,/^(/3) > define 


(4) 


(5) 




r( p + g + 1 ) ^ /p\ r(m + p + a + /3 + l) 

r(p + a + /3 + 1) 0 \m) r(TO+a+l) ^ 


ajb = 


(-)* 

klT(k + (x+l) 



(2p + a + p + l)r(p + A + a + /8 + l) 
(p - A;)!r(p + /3+1) 


Sp(,a, fi), 


( 6 ) 


f(x) = a:“(l - a:)'* 


Then fix) will satisfy (2) /or m = 0,1, • • • , w. 

2°. //, m addition 1°, Mn+i is known and a and ^ satisfy 


(7) 


>Sn+l(a, = 0, 


then fix) will satisfy (2) for m = n + 1 also. 

3®. //, in addition to and 2®, m »+2 is a ?80 known^ and if a, also satisfy 

( 8 ) Sn+2ia, 0) = 0 , 

then fix) will satisfy (2) for m n + 2as well. 

Proof. Let Pm'^\z) be the Jacobi polynomial of order m defined in terms 
of the hypergeometric function by 

(9) n-m, m + a + p + li<x + l-,i - Iz). 

Let — 2m) symbolically represent the expression gotten by substituting 

Mifc for Xk in the expansion of the polynomial (1 — 2x). There exist numbers 
Afn,q such that 


( 10 ) 


X 


= - 2x). 
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Also 

(11) /i.. = - 2/*). 

0 

For i?(o) > — l,fl(/8) > —1, define 

fM - r»n - (2p + « + 0 + l)pir(p + « + (8 + 1) 

( 12 ) ^ ^ ’ 0 r(p + a + i)r(p + /3 + 1) 

X - 2M)Pjr-'”(l - 2x). 

Then by (10), for m = 0,1, • • • , n, 

p ;+^.) -« 

X 11, A„., fz^a - x)'Pi“-'”(l - 2x)P‘r‘'’’(l - 2x) dx. 
0 Jo 

By the orthogonality of the Jacobi poljmomials, [1; §4.3], 

ff(x)x’^dx = 2:,4„.,Pi,“>'"(l - 2 m). 

Jo 0 

By (11), 

f f(x)x"' dx = ftm, (m = 0,1, • • •, n). 

Jo 

It follows from (2) that/(a:) as defined in (12) is the/(x) of (1). It remains to be 
shown that (12) may be expressed in the form (4)-(6). 

From (9), 

r(p + a + 1) 


Pi,‘’'”(l - 2x) 


(13) 


r(p + a + /S + 1) 

A (-)« r(p + m + o + |8+l) 
^ ^ ml(p - m)! r(m + ct + l) 


Py-^d - 2 m) =^,5,(a,/3). 


so by (4), 

(14) 

Inserting (13) and (14) into (12), 
f(z) - x(l x) Ep r(p + 3 + 1) 


xi* 


(~)* r(p + A; + g + /? + l) 15 j / ... 
A:!(P - A;)! m + a + 1) 


x"(l - xy E* 


(_)‘x* 


0 k\T(Jc + « 4 * 1 

>,/ V (2p + g + |3 + l)r(p + fe + «+ |9+l) 
^ r' (p - A) I r(p + i8 + 1) 
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= x“(l - 

0 

by (5), so the fix) of (12) may be expressed in the form (4)-(6), and part 1® of 
the theorem is established. 

If (7) holds, by (5), = 4"^ for A; = 0,1, • • • , n, and a = 0. There¬ 

fore, in (6), 

f(x) = x“(l - xf 

0 

and by part 1®, for the case in which n is replaced by w + 1, it follows that (2) 
holds for m = n + 1, so part 2® is established. The establishment of part 3® 
is essentially the same. 

In applying this theorem to the problem of empirical curve fitting, it follows 
from (6) that the constants a and should differ from zero only if the empirical 
curve approaches zero or infinity at one or both of its endpoints. With this 
in mind the following rules may be stated: 

Case A. If, in the empirical curve, /(O) 9 ^ 0 or cjo , and /(I) 5^ 0 or oo , set 
o£ = = 0, and let n be one less than the number of moments that it is desired 

to fit. 

Case B. If/(O) = 0 or 00 and /(I) 5^ 0 or 00 , set jS = 0 and determine a from 
(7), n being two less than the number of moments that it is desired to fit. 

Case C. If /(O) 0 or « and /(I) = 0 or co, set a = 0 and determine P 

from (7), n being two less than the number of moments that it is desired to fit. 

Case D. If /(O) = 0 or 00 and /(I) = 0 or 00 , determine both a and p from 
the two equations (7) and (8), n being three less than the number of moments 
that it is desired to fit. 

It may happen that these processes cannot be carried out, or at least cannot be 
conveniently carried out. If this is the case, aor p may be set arbitrarily and n 
taken as one unit higher than before, or both a and P may be set, and n taken 
as two units higher than before. 

In Case D, above, the solution of equations (7) and (8) may often prove 
difficult, making it advisable to follow the suggestions of the last paragraph. 
In certain special cases, however, their solution is not difficult. 

Suppose, for example, the moments satisfied the equations 

(15) Mm = w = 0, !,•••. 

If this is substituted into (4), and the order of summation reversed, on making 
use of the identity 

(-V - r(a)r(a - y + 1) 

^ 0 VP/r(p + y)^ ^ ^ r(a-y-n+l)r(n + y)' 


one obtains 
(17) 


5p(a, P) = i-)^SM a). 
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Therefore 

(18) /S 2 p+i(a, a) = 0. 

When n is an integer, either n + 1 or n + 2 is odd. Therefore when (15) 
holds, one of either (7) or (8) will be satisfied identically if we take = a. The 
other may then be solved for a. 

As an example, suppose one had the moments no = 1, mi = i, M 2 = ^, Ms = 

M 4 = iftAr, and wished to obtain an f(x) such that /(O) = 0, /(I) = 0. In this 
case n = 2, and (15) is satisfied. It follows that (7) is satisfied identically when 
P = a, and (8) gives 

r(2a + 5) . , r(2a + 6) /_l\ T(2a + 7) f 7\ 

T(a + 1) T(a + 2) V V r(a + 3) W 

^ , T(2a + 8) 3\ , r(2a + 9) / 31\ _ ^ 

r(a + 4) V 16/ r(a + 5) \240/ ‘ 

This easily reduces to 

_ a + 5/2 (« + 5/2)(a + 3) 

a + 1 (a + l)(a + 2) 

_ (g + 5/2)(a + 7/2) ^ (a + 5/2)(« + 7/2) ^ 

(g -j- l)(g ’‘h 2) 240 (g “|- l)(of "H 2) ^ 

which reduces to the quadratic 

4g^ - 6a + 5 = 0, 

from which 

(19) a = = 3/4 rfc: (1/4) VU^*. 

These may be substituted into (4)“(6) to complete the solution. 
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CONSISTENCY OF SEQUENTIAL BINOMIAL ESTIMATES 

By J. Wolfowitz 
Columbia University 

The notion of consistency of an estimate, introduced by R. A. Fisher, applies 
to a sequence of estimates which converge stochastically, with boundlessly 
increasing sample siee, to the parameter (or parameters) being estimated. Each 
estimate is a function of a sample of observations, the number in each sample 
being determined independently of the observations themselves. In sequential 
estimation, on the other hand, the number of observations is itself a chance 
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variable, determined by the sequence of observations and the application to 
them of a rule which may be part of a sequential test. In what follows we 
shall consider that the operation of sequential estimation is associated with a 
sequential test.^ 

The advantage of using consistent estimates is such as to suggest extension 
of the idea of consistency to sequential estimation. In the present paper we 
shall be concerned only with the estimation of a binomial probability (p, say). 
The obvious extension is that a sequence of estimates, each with its associated 
test, is consistent if the estimates converge stochastically to p. 

Since the number of observations required by a sequential test is a chance 
variable, a parallel to the classical sequence of samples of increasing size would 
be a sequence of sequential tests whose average (in some sense) sample sizes 
increase without limit. It seems reasonable to associate only such a sequence 
of estimates with this sequence of tests as ^vill converge stochastically to p, 
i.e., be consistent. 

Let 2 be a chance variable which takes the distinct values Ci and C 2 with proba¬ 
bilities p, 0 < p < 1, and q = I — p, respectively. Let Zi, • • • , Zn be a sequence 
of independent observations on 2 which terminates with the nth according to the 
specific sequential test under consideration. Denote by x and p, respectively, 
the number of observations C 2 and Ci in this sequence. Then a;, y and n = x + p 
are all chance variables. The couple g = (x, y) is called a boundary point of 
index n (see [1]). The sequence of observations which terminates at g is called a 
path. Let k{g) denote the number of paths which terminate at gf, and let fc*(p) 
denote the number of these paths whose first observation is c \. The ‘‘points” 
on the various paths together with all the points g constitute the “region” under 
discussion. 

Let P{n = y) denote the probability of the relation in braces. If 

f:Pj„=ji = 1, 

the region is called closed. Only closed regions will be considered below, so that 
this assumption will henceforth be made without explicit formulation. It has 
been shown by Girshick, Mosteller, and Savage [1], that p{jg) = k*{g)/k{g) 
is an unbiased estimate of p for any closed region Ry i.e., 

2 p(ff)A:(?)pY - P. 

where the summation takes place over all the boundary points g of R, For 
many important regions this estimate is the unique unbiased estimate. 

Let there be given an infinite sequence of sequential tests with each of which 
we associate the estimate p(g). Consider the ith one of these, and let no; be 
the smallest number of observations required for a decision, i.e., no< is the smallest 

^ Really all that is required is a rule for terminating the observations such that its region 
R is closed (see below). However, we defer to conventional statistical usage in referring 
to “tests.*' 
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value of j for which P[n = j] =t= 0. The theorem proved below asserts that if 
no» approaches infinity with i the estimate p{g) converges stochastically to p. 
To put it in other words: if Ti, T 2 , • • • is the sequence of tests, and €1 and €2 
are arbitrarily small positive numbers, there exists a positive number ./(ci, € 2 ) 
such that, for all T, such that i > J, 

^'’11 vig) - p I > «il < ‘ 2 , 

when no» —> . An important example of such a sequence is that of the Wald 
sequential binomial tests [2] obtained as follows: Let ai, a 2 , * • • , a, • • • and 
/3i, ft , • • • , i8» • • • , be two sequences of positive numbers all of which are less 
than J and wliich approach zero as i > cc. Let po and pi, 0 < po < pi < 1, 
be two fixed numbers, 

c, = log^, C 2 = log^j- Z, = 'Ezt. 

Po (1 — Po) ifc -1 

Finally let the rule for terminating the process of drawing observations be as 
follow’s for the zth test 7\ : The process of draw ing observations terminates at 
the smallest integer n for w hich either 

Zn > log -- - or Zn < log -. 

a, 1 — a. 

Since (1 — iSj/a, —^ 00 and ^t/{l — a») —> 0 while c*i and C 2 are constant, it is 
evident that the hypothesis of the theorem is satisfied. 

The property of being unbiased is not generally considered an indispensable 
characteristic of an optimum estimate, while consistency is generally so regarded. 
Our theorem shows that p(g) enjoys the latter property with respect to important 
sequences of sequential tests. 

Theouem: Let Ti , • • • , T*, • • • a sequence of sequential binomial tests. 
For the ith test 1\ let no^ be the smallest integer such that P{n = aio»} =t= 0. Finally 
let not —> Qo as i co. Then p{g) converges stochastically to p as i x . 

Proof: For tyiiographic simplicity w^e shall use as the designation of the 
generic element of the sequence noi, no 2 , ••• . No confusion will be caused 
thereby. 

Let n' = no — 1, and > 0 and ^2 > 0 be arbitrarily small fixed numbers. 
Let k'{g) be the number of paths w'hich end at the point g and are such that 
I y'/n' — p I < 5i, where y' is the number of observations Ci among the first n' 
observations. We then have 

Lemma 1. For no suffixnently large 

(1) E k’(g)pY > I- Sz 

where B is the set of boundary points of R. 

Proof: Consider the totality [h] of all points h = (a', p'), with x' + p' = n'. 
Here and i/' denote, respectively, the number of observations C 2 and Ci in the 
sequence of the first n' observations on z. Let ko{h) denote the number of paths 
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to h. Let C denote the set of points h such that | y'/n' — p ( < 5i. If n© is 
large enough we have, by the law of large numbers, 

E >1-32. 

htc 

Let k{hy g) be the number of paths from h to g. From Theorem 2' of [3] it 
follows that 

(A) S Hh, g)p*^ = p'V'. 

a*H 

Also from the definitions of the various symbols involved it readily follows that 

= S ko(h)k(h, fit). 

heC 

Hence 

i: = E (E ko{h)k(h, g))py = E (E Uh)k(.h, g)pY) 

gtB gtB h*C gtB htC 

= E h{h)(£Hh,g)pY) = E h{h)p''Y > l - «». 

htC gtB htC 

This proves Lemma 1. 

Let ^(g) == [k(g) — k^{g)]k{g). Thus ((g) is a chance variable, being a function 
of the chance point g. 

Lemma 2. . Let 6© and Bi be arbitrarily small positive numbers. For n© suffidenUy 
large 

(2) P{((g) ^ 58} > 1 - 34. 

Proof: If (2) were not true, we would have 

(3) ^ = s fc'((7)pV < (1 - «4) + (1 - = 1 - S,«4. 

Kg) 

Choose the 32 of Lemma 1 so that 32 < 3834 . For some large value of n© we 
would then have a contradiction between (1) and (3). This proves the lemma. 

Let g be any boundary point. Consider any path whose y' is such that 
I y'/n' — p 1 < 3i ; let us call such a path one of type T. Consider the terminal 
sequence S of this path, 

^ 9 ZltQ+l 9 * * * > 2Jn 

This sequence, together with g = (x, y), uniquely determines y\ Any permuta¬ 
tion of y' elements Ci and n' — y' = x' elements C 2 may serve as the initial sequence 
of n' observations of a path which terminates at g and has the terminal sequence 
S. For no boundary point is of index smaller than n©, so that under permuta¬ 
tion of the first n' observations a path remains a path, i.e., the process of taking 
observations will not terminate prematurely as a result of the permuting of the 
elements. Of these permutations a proportion y*/n* begin with the element Ci. 
We deal in this manner with all the different terminal sequences of the paths of 
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type T which end at g. Let k*\g) be the number of these which begin with Ci. 
We obtain 

Lemma 3 . For all g such that k'{g) =t= 0 


k*\g) 

k'(g) 



< 6i. 


Putting Lemmas 2 and 3 together we have 

Lemma 4 . As no oo, k*'{g)/k{g) converges stochastically to p. 

Now it follows in a manner similar to that of Lemma 2 that, as no oo, 
k*'{g)/k*{g) converges stochastically to one. This, together with Lemma 4, 
proves the theorem. 
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BOOK REVIEWS 

Mathematical Methods of Statistics. HaraM Cramer. Uppsala, Sweden: 

Almqvist and Wikscll, 1945. pp. xvi, 575. (Princeton, N. J.: Princeton 

University Press, 194G. $6.00) 

PkVIEWED BY Will Feller 
Cornell University 

This book represents a contribution of a novel kind to the statistical literature 
and will render valuable services both as textbook and reference book. Of its 
three parts the first one (134 pages) is entitled Mathematical Introduction and 
develops the necessary formal mathematical tools. The second part (186 pages) 
is devoted to Random Variables and Probability Distributions, that is to say, to a 
chapter of the modern theory of probability. The third, and main, part of the 
book (some 233 pages) is entitled Statistical Inference. Ordinarily these three 
topics Nvould require consultation of three or more books, and these would rarely 
be found on the same shelf. However, the masterly exposition succeeds in creat¬ 
ing the impression of natural unity and harmony. The ideas are developed with 
elegance and apparent ease as if the line of presentation followed a well explored 
path. The uninitiated will not notice how unconventional the treatment is and 
how the very selection of topics depends on the author's scientific personality. 

It is hardly necessary to point out that Cramer's book fills an urgent need. 
The emergence of statistical theory and methodology as an exact science, firmly 
grounded in mathematical probability, is only of recent date. Its rapid develop¬ 
ment >vent hand in hand with an extraordinary increase of the number and im¬ 
portance of its various applications. Under such circumstances there was 
naturally little time for an exposition of the theoretical foundations and ramifi¬ 
cations. ^Modern statistical inference has its roots in the classical limit theo¬ 
rems of probability. Now classical probability used to consist of a bewildering 
collection of special and mutually uncorrelated problems; unified guiding princi¬ 
ples and methods are a rather new development and have not yet found expression 
in the textbook literature. The original investigations are usually written in an 
exceedingly abstract language and the existing close ties to applications are not 
apparent. Consequently, there is no easy access either to probability or statis¬ 
tics and it is often diflScult to establish whether, or to what extent, various asser¬ 
tions have actually been proved. The present book therefore closes a serious 
gap in the literature and will greatly facilitate both teaching and research. 

Of the 12 chapters of the Mathematical Introduction 9 are devoted to the theory 
of measure and integration. The antiquated theory of the so-called Riemann 
integral (kept alive by elementary textbooks) considered only point functions 
y = /(ic), where the independent variable is a point. The temperature at a 
given point or the velocity at a given moment arc typical examples. Many 
mathematical considerations simplify greatly if from the very begiiming also set 

186 
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functions y = F(A) are introduced, where the independent variable is a set. 
Typical examples are mass in mechanics, the amount of heat or of electricity, 
area or wealth of a geographic region, and the probability of events (i.e. sets in 
sample space). The Ixibesgue-Stieltjes theory frees the concept of integral from 
artificial devices and reduces it to the natural notion of mean values with respect 
to set functions. In a simile, believed to be due to Lebesgue, the Riemann inte¬ 
gral corresponds to the procedure of a grocer who computes the day's receipts 
by actually adding the several amounts in the order as they had come in. The 
Lebesgue procedure imitates tlie more intelligent grocer who orders his cash in 
piles of notes and coins of e(iual denomination and counts them. The analogy 
with the customary procedure of computing mathematical expectation is clear. 
The Ijebesgue-Stieltjes integral is conceptually simpler than the Riemann integral 
and can be presented in as simple a way with rigor adequate for elementary text¬ 
books. It has become an indispensable tool in probability, statistics, physics, 
and other applied fields. Since it has, unfortunately, not found its way into 
calculus textbooks, physicists are compelled to use the less flexible notion of the 
Dirac ^-function, and the formal mathematical apparatus in general becomes 
unnecessarily clumsy. It is a curious anomaly that so many calculus textbooks 
profess to lx‘ written with a view' to applications and yet completely disregard 
the most obvious practical needs and that the teaching of practical mathematics 
should remain uninfluenced by the great developments of the last fifty years. 

In such (‘ircumstances the* chapters on integration will be particularly w’elcomc 
to statisticians as probably the only place in the literature w’here they will find 
easy access to the theory. Of course, this exposition leads far beyond w'hat the 
average* statistician will reepiire under ordinary circumstances and beyond the 
necessary prerequisites of the main body of the book. Of the 88 pages roughly 
half can be omitted at first reading in accordance with detailed instructions given 
in the Preface. The remaining half will form a valuable reference book for 
theorems and tools used occasionally in connection with more delicate parts of 
statistical theory. The mathematical introduction contains also a chapter on 
Fourier integrals (characteristic functions), one on matrices and (quadratic forms, 
and finally miscellaneous complements such as orthogonal polynomials, Euler's 
summation formula, beta and gamma functions, etc. 

The title to the second part, Random Variables and Probability Distributions, 
is the same as that of the author's well-known Cambridge Tract of 1937. Both 
start with a discussion of the foundations along axiomatic lines. The new' treat¬ 
ment does not differ essentially from the old one, but some changes are intro¬ 
duced which are regrettable in the reviewer's opinion (in particular axiom 3). 
Otherwise there is practically no overlap between the tw o expositions. The 1937 
booklet devoted much si)ace to the asymptotic expansions connected with the 
central limit theorem which are due to the author himself. This topic is not 
touched upon in the present book. This is a judicious procedure since the 1937- 
booklet is generally accessible (although at present sold out). Instead we now 
find a detailed study of some univariate distributions such as x^ Student's t. 
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Fisher's z, the Pearson system, etc., none of which were mentioned in the Cam¬ 
bridge tract. Similarly, there is now a section on correlation and regression, 
and the normal distributions in several variables. The theory of probability is 
developed only to the extent of the formal theory of distribution functions. This 
implies that even so important a notion as stochastic convergence is treated only 
summarily while the strong law of large numbers falls completely outside the 
framework of the book. This is regrettable inasmuch as the strong law is of 
greater importance than the classical weak law (whose fame rests essentially on 
a classical misunderstanding). It should be mentioned that this second part of 
the book contains some 39 well chosen illustrative exercises the solution of which 
is left to the reader. 

In the main part of the book, entitled Statistical Inference^ the outer form 
changes inasmuch as the text there is accompanied by numerous practical exam¬ 
ples. However, the exposition remains mathematical in nature and the main 
emphasis rests on exact formulations; much attention is paid to the establishment 
of the precise conditions of validity of the individual theorems, their logical 
interrelations and their connections with general probabihty. The expert will 
find many minor and major improvements in formulations and proofs. They are 
too numerous to be listed here. Suffice it to point out, as a typical example, the 
theorem on pp. 426-27 concerning the limiting form of the distribution with 
estimated parameters; this theorem appears to be more general than usually 
stated and also the proof seems to be novel. The topics treated in the statistical 
part of the book will be seen from the following list of titles to the chapters. 25. 
Preliminary Notions on Sampling. 26. Statistical Inference (general orienta¬ 
tion). 27. Characteristics of Sampling Distributions (moments, semi-invariants, 
corrections for grouping, etc.). 28. Asymptotic Properties of Sampling Dis¬ 
tributions (moments, extreme values, range, etc.). 29. Exact Sampling Distri¬ 
butions (degrees of freedom. Student, Fisher, correlation and regression coefii- 
cients, partial and multiple correlations, generalized variance, etc.). 30. Tests 
of Goodness of Fit and Allied Tests (treating mostly applications of x^)- 31. 
Tests of Significance for Parameters. 32. Classification of Estimates (sufficient, 
efficient and asymptotically efficient estimates; minimum variance, etc.). 33. 
Methods of Estimation (method of moments, maximum likelihood, x^-niinimum 
methods). 34. Confidence Regions. 35. General Theory of Testing Statistical 
Hypotheses. 36. Analysis of Variance. 37. Some Regression Problems. There 
follow tables of the normal distribution, the x^ aiid the t-distributions, and a long 
fist of references. 

If an expression of wishes for a second edition were permitted, most statisti¬ 
cians would probably give first choice to non-parametric and sequential tests. 
It is needless to point out that the latter became public only after completion of 
the Swedish edition of the present book 

Even this short account will show the extremely wide range of topics and 
theories covered in the book, from abstract integration to randomized experi¬ 
ments. They are all presented with uniform lucidity. The exposition through- 
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out is formal, and yet inspiring, rigorous and yet never pedantic. It will serve 
as an example worthy of imitation and is an achievement on which the author 
deserves our sincere congratulations. 


The Advanced Theory of Statistics. Vols. I and II. Maurice G. Kendall 
London: C. Griffin and Co., Ltd. Vol. I. Second ed. revised, 1945; pp. xii, 
457, 50 shillings. Vol. II. 1946; pp. viii, 521; 42 shillings. 

Reviewed by M. S. Bartlei't 
Cambridge University and The University of North Carolina 

With the recent appearance of the second volume, it is now possible to review 
as one work this comprehensive treatise. To quoto the author’s opening re¬ 
marks to the Preface to Volume I: “The need for a thorough exposition of the 
theory of statistics has been repeatedly emphasized in recent years. The object 
of this book is to develop a systematic treatment of that theory as it exists at the 
[)roscnt time.” An outline of the contents, which in the two volumes make up 
just on a thousand pages, will indicate that this formidable task has been squarely 
faced by the author, who, when a tentative co-operative venture of writing such 
a treatise wius upset by the outbreak of the war, continued alone with the project. 

Volume I contains sixteen chapters. The first six introduce the concept of 
fretpiency distributions via observational data on groups and aggregates, and 
their mathematical representation (Ch. 1), measures of location and dispersion 
(Ch. 2) and moments and cumiilants in general (Ch. 3), characteristic functions 
(Ch. 4), and ending with a description of the standard distribution functions, such 
as the binomial, Poisson, hypergeometric and normal distributions, and the 
Pearson and Gram-Charlier systems. The next section opens with probability 
(Ch. 7) and proceeds to sampling theory (Chs. 8-11), including a chapter (Ch. 10) 
on exact sampling distributions, many of the standard sampling distributions 
being used in this chapter to illustrate the mathematical methods available for 
obtaining sampling distributions. Chapter 11 deals with the general sampling 
theory of cumulants, including a useful reference list of formulae and a demon¬ 
stration, due to the author, of the validity of Fisher’s combinatorial rules for 
obtaining these formulas. The section concludes with a chapter on the Chi- 
square distribution and some of its applications. The last four chapters of 
Volume I deal with association and contingency, correlation, including partial 
and multiple correlation, and rank correlation; this last chapter being a compre¬ 
hensive treatment including comparatively recent results of the author. 

It will be convenient to list also the contents of Volume II before any critical 
comment on either volume. The first section of the second volume comprises 
four chapters on the theory of estimation, including a derivation of the properties 
of the maximum likelihood estimate (Ch. 17) and separate chapters on Fisher’s 
theory of fiducial probability and Ne 3 nnan’s theory of confidence intervals. The 
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second main section, according to the author’s remarks in the preface to A^olume 
II, deals with the theory of statistical tests and comprises chapters 21, 23, 24, 20, 
27 and 28; of these after an introductory chapter (Ch. 21) on tests of significance, 
chapters 23 and 24 cover analysis of variance, chapters 26 and 27 give a fairly 
detailed account of the general theory of significance-tests originated by Neyman 
and Pearson and Chapter 28 deals with the recently developed techniques of 
multivariate analysis. The remaining chapters are 22 on regression, 25 on the 
design of sampling enquiries, and Chapters 29 and 30 on time-series, another 
subject in which the author has himself taken an active interest. Finally, there 
are two appendices, A consisting of a few addenda to Volume I, and B an exten¬ 
sive bibliography of theoretical statistical papers. 

The volumes are attractively printed; and each chapter concludes with a useful 
collection of examples for the reader. 

In any comprehensive treatment of a wide subject there can be no clearly de¬ 
fined order of presentation; nevertheless, the author’s order of chapters in Volume 
II and in particular his inclusion of analysis of variance among the chapters on the 
theory of statistical tests is a little puzzling, and the reviewer’s preference would 
have been to see this important subject treated earlier, together with regression 
analysis, and their link with the classical method of least squares more firmly 
outlined. Incidentally, there appears to be no mentirm of the Fourier analysis 
of observational data except in its relation to periodogram analysis (Ch. 30). 
This change of order w ould perhaps also have allow ed a shift forward of Chapter 
25 on the design of sampling enquiries, and a more compact section on multiple 
correlation, culminating with the chapter on multivariate analysis before the 
chapters on the general theory of statistical inference were begun. 

Another arrangement of rather doubtful value in Volume II is the allocation of 
separate chapters to fiducial probability and to the theory of confidence inter¬ 
vals. The problem of how to deal with a field which is still a battleground is 
admittedly not an easy one, and this particular one is an embarrassment at 
present to many teachers, but it may be questioned w^hether strict impartiality 
is the best answer. To take a hypothetical example, there would seem to be no 
particular virtue in a textbook which expounde<l, in parallel, statistical methods 
of inference using direct probabilities and the method of “inverse probability”, 
leaving the reader to decide at the end which he should adopt. 

The most criticizable arrangement, however, occurs in Volume I with the late 
and rather scanty treatment of probability in Chapter 7. To begin with ex¬ 
amples of statistical data is sound, but since the w hole conceptual model erected 
to deal with such data is based on probability theory, it does not seem sufficient 
for a reader who “feels keenly on the subject” to do as the author suggests in the 
Preface and read Chapters 7 and 8 after Chapter 1. Even if he does so, he will 
find no very clear exposition of the statistical theory of probability,—no mention, 
for example, of the law s of large numbers, whether for simple dichotomies or for 
entire continuous distribution functions, that show how the conceptual model 
adequately corresponds with the empirical notions of “in the long run” or “for 
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a large enough sample”. The actual arrangement, moreover, leads to an ap¬ 
parently rather arbitrary * treatment of theorems on limiting distributions; 
the First Limit Theorem, which deals with the equivalence of the limits of dis¬ 
tribution function and corresponding characteristic function sequences, is given 
in the chapter on characteristic functions (Ch. 4), and the Central Limit Theo¬ 
rem, dealing with the convergence to normality of a sum of n independent random 
variables, is given in the chapter on probability. 

In the proof of the second part of the First Limit Theorem, dealing with the 
conditions under which a sequence 0n(O of characteristic functions determine 
the limiting distribution function F (x), the author has not yet corrected an error 
that occurred in Cramer’s original version, which Kendall follows (section 4.12). 
Correct conditions for convergence of the distribution function sequence Fn(x) to 
F(x) (at all continuity points of F) are convergence of the characteristic function 
sequence to for all real uniformly in at least some finite t interval (cf. H. 
Scheff4, Math. Reviews, Vol. 6 (1945), p. 89). 

Another proof in Volume I which appears to need clarification is the geometri¬ 
cal derivation of the distribution of the multiple correlation coefficient in the case 
of a non-zero true correlation (section 15.21). The blunt statement is made, 
following equation (15.51), that the sample correlation coefficient R and an angle 
^ (defined in the text) are independent, a statement which is incorrect. How¬ 
ever, if the logic of Fisher’s original derivation is examined, it turns out that the 
relation of R and ^ is only required when the true correlation is zero; under such 
conditions R and ^ are independent. 

In Volume II there is a sentence requiring correction and amplification in the 
derivation (in the case of zero true canonical correlations) of the sampling canoni¬ 
cal correlation distribution (section 28.30). The sentence “Consider the dis¬ 
tribution for a given value of /,/ and • • • ” should be corrected to read “Con¬ 
sider the distribution for a given value of Uj + Ziy • • • ”. Some justification that 
the flistribution is independent of Uj -1- zy, is then still needed. 

There is inevitably, owing to the time the book was written, no mention of 
sequential analysis, the sampling technique developed during the war by Wald 
and others and only recently “derestricted”. Again, in chapter 18, where the 
^vork of Aitken and Silverstone on unbiased estimates with minimum variance is 
referred to, the simple inequality connecting the variance of any unbiased esti¬ 
mate with Fisher’s information function throws an interesting new light on this 
aspect of the estimation problem (see, for example, H. Cramer, Mathematical 
Methods of Statistics, section 32.3, or C. R. Rao, Bulletin Calcutta Math. Soc., 
Vol. 37 (1945), p. 81), but was not known to the author when this chapter was 
written. Such omissions are merely an indication of the developing nature of the 
subject, and it is hoped they can be remedied in later editions. There is, how¬ 
ever, especially in Volume II, an occasional impression of patchiness in the treat¬ 
ment not altogether excusable on such grounds. This can perhaps be illustrated 
from the last chapter, a valuable contribution to the still-growing subject of 
time-series, but where the importance of some known results does not always 
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seem sufficiently stressed; in particular, the Wiener-Khintchine relation between 
the periodogram and correlogram is noted (section 30.68) as *^an interesting re¬ 
lation’’, whereas it is a fundamental relation in the modem method of approach 
to time-series, giving much deeper insight into the correct interpretation of 
classical periodogram analysis. 

These criticisms, which could be extended to cover minor errors and mis¬ 
prints, are not intended to detract seriously from what is a remarkable achieve¬ 
ment. An excellent sense of proportion has been maintained throughout be¬ 
tween mathematical theory and illustrative discussion and examples. This makes 
this treatise, if both the breadth and level of the subject matter are taken into 
account, at present unique. It will be an indispensable reference book to every 
teacher and advanced student of the theory of statistics. 


Sequential Analysis of Statistical Data: Applications. Prepared by the 
Statistical Research Group, Columbia University for the Applied Mathe¬ 
matics Panel, National Defense Research Committee, Office of Scientific Re¬ 
search and Development. SRG Report 255, Revised; AMP Report 30.2R, 
Revised. New York: Columbia Univei*sity Press, September 1945. pp. vii, 
17; iv, 80; v, 57; iii, 25; iii, 18; iii, 39; ii, 41. $6.25. (London: Oxford Uni¬ 
versity Press, 1946.) 


Reviewed by John W. Tukey 
Princeton University 

Many of the features of this compendium are familiar to most of the readers of 
this review, but for the benefit of the others I shall enumerate them briefly. It 
consists of a heavy looseleaf binder containing 7 booklets of distinctive colors— 
each saddle stitched and usable separately. It is the last word (to date) in pre¬ 
senting sequential analysis to the statistician who may wish to use it in practice. 
It covers five elementary cases (each in a booklet, the two others being used for 
introduction and appendices): 

Acceptance or rejection by percent defective (Sec. 2) 

Comparative percent satisfactory (Double dichotomy) (Sec. 3) 

Acceptance or rejection by the adequacy of the mean (with known variability) 
(Sec. 4) 

Acceptance or rejection by the exact value of the mean (with known variability) 
(Sec. 5) 

Acceptance or rejection by the smallness of the variability (Sec. 6) 

These cases are covered in complete detail, with illustrative examples, tables and 
charts. A copy should be accessible to every teacher of statistics and to every 
statistician in industry or experimental work who can propose new techniques of 
testing. 
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With this general introduction let us go on and explain what the reader will 
not find and what further work in this line the reviewer awaits mth keen interest. 
The classical testing procedure was to test a sample of predetermined size and 
then decide to accept or reject. Long ago curtailed sampling and double samp¬ 
ling were developed to cut comers legitimately and reduce inspection costs. 
There are two situations, each more frequent in war than in peace, where it is 
clearly desirable to reduce the average number of items tested to a minimum: 

(I) Where essentially all lots are accepted and the test is destructive so that the 
items tested are the main loss of production, or 
(II) Where the cost of testing an item is large in comparison with the cost of 
production. 

Subject to a practically unimportant allowance for the finite size of the lots, and 
to an allowance of unknown importance for the quality of lots presented, the 
methods of sequential analysis minimize this average number among all methods 
so far considered. Wlien situation (I) or (II) holds without modifiying complica¬ 
tions, then, the best known method is sequential analysis, the natural descendant 
of double sampling. Otherwise, the situation is far from clear, and much judge¬ 
ment is involved in setting up a practically efficient scheme. The reader will 
get no help on this problem of judgement, nor in the problem of setting risks from 
the book under review—he will get every needed help with the mathematical 
problem of setting up a secpiential plan to meet chosen risks, including complete 
tables of all necessary functions, including natural logarithms. 

There is no reason to suppose that sequential analysis is the last word in testing 
procedures for the general problem of efficient testing, but what should be the 
next step ahead is not a step for the mathematical statistician. What is needed 
now is a careful analysis, by the operational research techniques so useful during 
the war, of a half-dozen industrial testing situations to determine what properties 
of the testing procedure are involved in cost and to what extent. Do we want 
the minimum average sample size, the minimum average square of the sample 
size—or what? With this there should go a corresponding operational study of 
the advantages of different OC curves, including those of what now seems to be 
a peculiar shape. Given these studies, we could put the problem in mathematical 
statistics to the mathematical statistician which he would then solve. But with 
the present lack of operational research groups in industry, it is probable that 
we will proceed in an unnatural way, and that the mathematical statistician will 
take the next step forward. For reasons of mathematical simplicity it is not 
unlikely that the sample plan with the minimum average squared sample size 
will come next. 

The credit for the book is clearly assigned on the inside cover of each pamphlet 
in the following words: “So many members of the Statistical Research Group 
(Columbia) have participated in the preparation of this report, a previous edi¬ 
tion of which was prepared by H. A. Freeman, that its authorship is attributed 
to the group as a whole. The responsibility for planning and preparing this 
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edition has been shared by H. A. Freeman, M. A. Gii-shick, and W. Allen Wallis, 
with the cooperation of Kenneth J. Arnold, Milton Friedman, Edward Paulson, 
and others. The theory of sequential analysis is mainly the work of A. Wald.'' 

It may be of interest to notice a few minor points for the record. On page 
1.01 it is indicated that 100% inspection is 100% effective— this seems far from 
industrial experience. Another badly needed set of operational studies would be 
on the influence of the sampling plan on inspector’s inspection. On page 2.27, 
the footnote suggests that when a tabular procedure is used instead of 
a graphic one, that more decimal places should be kept—the logic of this is not 
clear. On page 4.14 it is stated that ‘^similarlj^', if all patches had tested 400 
minutes, the experiment would have terminated at 9.4 ...”. Clearly no such 
experiment can terminate after a fractional number of tests. On page A.09 
it is stated that “Finally it should be mentioned that truncation of any kind 
ought generally to be avoided”. This seems to the reviewer to be a rash state¬ 
ment, for when not only average sample size but all other properties entering into 
the practical efficiency of a sampling plan are considered, this decision will almost 
certainly be reversed. The relatively small number of these detailed points is 
an evidence of careful and competent workmanship. 

A footnote to the Appendix (B) on some principles of sequential analysis states: 
“Any mathematician who may stray into this Appendix should be assured that 
the validity of the conclusions in no case depends upon the type of reasoning 
presented here; indeed, even for intuitive or heuristic arguments mathematicians 
may prefer those given in SRG 75”. This warning and caveat seems unduly 
strong—the appendix is recommended to all mathematically minded newcomers 
to sequential analysis. 

The same appendix warns the reader in a few places that the theory set forth 
does not allow for the fact that samples come in units. If the reader tries to 
apply the theory to cases far from normal inspection practice, for example with 
risks of 0.25 and average sample sizes of 12, he will then find out that this does 
occasionally make a difference. In conventional circumstances the approxima¬ 
tion will not bother him. 



NEWS AND NOTICES 

Readers are irwiied to submit to the Secretary of the Institute news items of interest 

Personal Items 

Mr. Kurt W. Back has accepted a position with the Research Center for Group 
Dynamics, Massachusetts Institute of Technology. 

Mr. Stanley D. Canter was discharged from the Army in October and has been 
enrolled as a graduate student in mathematical statistics at Columbia University. 

Mr. William W. Cooper has accepted a position at Carnegie Institute of Tech¬ 
nology, Pittsburgh. 

Mr. Robert Dorfman is enrolled as a graduate student in the Department of 
Economics, University of California, Berkeley, and is also serving as a teaching 
assistant in that department. 

Dr. Nicholas Fattu, formerly at Michigan State College, has accepted a teach¬ 
ing position at Indiana University, Bloomington. 

Mr. John P. Gill is now Chief of the Research and Progress Analysis Division, 
War Assets Administration, Houston Regional Office, Texas. 

Dr. Clausin D. Hadley has accepted a position with the Graduate School of 
Business, Stanford University. 

Mr. Malcolm H. Henry is now Assistant Statistician in the Statistical Depart¬ 
ment of the Michigan State Department of Social Welfare, Lansing. 

Dr. Alston S. Householder has accepted a position as Principal Physicist with 
the Monsanto Chemical Company, Clinton Laboratories, Oak Ridge, Tennessee. 

Mr. Morton Kramer is now with the Office of International Health Relations, 
U. S. Public Health Service, Washington. 

Mr. F. C. Leone, who was discharged from the military service in the fall, has 
returned to his former posisiton in the Department of Mathematics at Purdue 
University, Lafayette, Indiana. 

Mr. Philip J. McCarthy, formerly at Princeton University, is now at Cornell 
University, Ithaca, New York. 

Mr. Edward C. Molina has been named special lecturer in Mathematics at 
Newark College of Engineering, in addition to Dr. Emil J. Gumbel, previously 
mentioned. 

Mr. Nicholas Pastore has accepted a position in the Department of Mathe¬ 
matics, City College of New York. 

Dr. William S. Robinson is now Assistant Professor of Sociology and Statistics, 
University of California at Los Angeles. 

Dr. Leonard J. Savage, who has a Special Rockefeller Fellowship, is spending 
the academic year at the Institute of Radiobiology and Biophysics, University of 
Chicago. 
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Professor Dunham Jackson died at Minneapolis on November 6, 1946. From 
1919 until 1946 Mr. Jackson was Professor of Mathematics at the University of 
Minnesota, and in 1946 was named Professor Emeritus. 

Professor Charles C. Wagner died suddenly on May 23, 1946, at the age of 52. 
He was acting dean of the College of Liberal Arts of Pennsylvania State College 
when he died. 


Those interested in the work of the Mathematical Tables Project, will, upon 
request, be placed on the mailing list for copies of the monthly progress reports, 
issued by the Project. Requests should be addressed to Dr. Arnold N. Lowan, 
150 Nassau St., New York, N. Y. 


Statistical Research Laboratory, University of Michigan 

Several developments in instruction and research in the general field of sta¬ 
tistics are in progress at the University of Michigan. 

At the beginning of the current academic year the new Statistical Research 
Laboratory was opened. It is planned that this unit, which is a division of the 
Graduate School, will serve as the center for research employing statistical me¬ 
thods and for research in statistical methodology. Free consultation and advice 
on statistical matters are offered to all members of the University engaged in 
research and the latest types of computing machines are available for their use at 
no cost to them. Or the Laboratory will undertake, at fees to cover costs, com¬ 
puting and the analysis of data for such individuals or units of the University. 
The Laboratory will have available the services of the University's completely 
equipped Sorting and Tabulating Station and expects to continue to provide a 
center for the most efficient computing service as improved machines are de¬ 
veloped. The technical assistants employed by the Laboratory will be advanced 
students of statistics who will thus have the opportunity to supplement their 
training with experience with actual statistical investigations. Professor 
C. C. Craig as Director and Professor P* S. Dwyer are in charge of the new labora¬ 
tory, each on a half-time basis. 

The new Laboratory is a research and not a teaching unit and is distinct from 
the large statistical laboratories for the use of students in statistics courses already 
in existence on the campus. With respect to instruction in theoretical statistics, 
the curriculum in that subject in the Mathematics Department has recently been 
revised and extended to include twenty-four semester hours at the undergraduate 
and graduate levels in addition to courses in probability, finite differences, 
graphical methods, and quality control. The somewhat related professional 
program in actuarial mathematics has likewise been strengthened. The teaching 
staff for these two curricula includes Professors H. C. Carver, A. H. Copeland, 
C. C. Craig, P. S. Dwyer, C. H. Fischer, and C. J. Nesbitt. 

A number of postwar research programs whose pursuit involves the use of 
probability and statistical methods have been established at the University of 
Michigan. Of especial interest is the new Survey Research Center under the 
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leadership of Professors R. Likert and A. A. Campbell who will continue activities 
begun by their group in Washington in the Department of Agriculture. Re¬ 
search by survey methods in the social sciences for public and private agencies 
and in survey methods themselves will be pursued and in addition a training 
program combining formal courses and apprenticeship in the Center is being 
set up. 


New Members 

Tlie following persons have been elected to membership in the Institute: 

Albert, GeorgeE., Ph.D. (Wisconsin) Head, Mathematics Division, Research Dept., Naval 
Ordnance Plant, Indianapolis, Ind., IIO 4 N. Oakland Ave. 

Ament, Richard P., B.A. (Cornell) Scientific Aid, 2129 20lh St., N., Arlington, Va. 

Bennett, Myra S., (Mrs. C. A.). A.B. (Michigan) Office Mgr., Institute of Math. Stat., 
Rackham Bldg., Ann Arbor, Mich., P. 0. Box JUS, Saline. 

Blankmeyer, Edith., A.B. (Western College) Stat., Res. Dept., National Broadcasting Co., 
30 Rockefeller Plaza, New York 20, N. Y. 

Blyth, Colin, Jr., M.A. (Queen’s Univ. and Univ. of Toronto) Graduate student, Univ. of 
N. Car., Chapel Hill, N. Car., 209 Mangum Dormitory. 

Brown, Philip, B.S. (Pittsburgh) Stat., R. 329 Standard Oil Bldg., 3rd and Constitution 
Aves., Washington, D. C. 

Bruno, O. P., B.M.E. (New York Univ.) Chief, Methods Section, Ballistic Research Labs., 
Aberdeen Proving Ground, Md. 

Carrier, Norman H., M.A. (Cantab) Civil Servant, Mathematical Statistics Section of 
Chief Scientific Advisers Division, Ministry of Works, c/o Westminster Bank, Palmers' 
Green, N. IS, London, England. 

Chand, Uttam, M.A. (Punjab Univ., India) Graduate student, Univ. of N. Car., Chapel 
Hill, N. Car., 112 Mangum Dormitory. 

Crow, Edwin L., Ph.D. (Wisconsin) Mathematician, Science Dept., Res., Devel., and Test 
Organization, USNOTS, Inyokern, Calif. 

Dang, Mary., M.A. (California) Graduate student, Columbia University, New York 27, 
N. Y., Box 267, Johnson Hall, 

Ens, Catherine C., B.S. (Dayton) Stat. Res. Ass’t, Graduate School, Ohio State Univer¬ 
sity, Columbus, Ohio, 267 Fifteenth Ave.^ Columbus 10, 

Fox, William H., Ph.D. (Indiana) Ass’t Prof, of Educ. and Ass’t Director of Res. and 
Field Service, Indiana Univ., Bloomington, Ind., 729 E. Hunter, 

Geisler, Murray A., M.A. (Columbia) Operations Analyst, Headquarters Army Air 
Forces, 222 N, Piedmont St., Arlington, Va. 

Gershenson, Charles P., B.B.A. (C.C.N.Y.) Res. Assoc., Institute of Psychological Res., 
Box ISO, Teachers College, New York 27, N. Y. 

Gilford, Leon, A.B. (Brooklyn) Econ. Analyst, Census Bureau, Washington, D. C., 141 O 
19th St., S. E. 

Goudsmit, S., A., Ph.D. (Leyden) Prof, of Physics, Northwestern Univ., Evanston, III. 

Halperin, Max, M.S. (Iowa) Graduate student, Univ. of N. Car., Chapel Hill, N. Car., 211 
No, Columbia, 

Halperin, Sidney L., Ph.D. (Ohio State) Psychologist, Neuropsychiatric Institute, Univ. of 
Mich. Hospital, Ann Arbor, Mich., 2j^ 1 Pittsfield Blvd., Pittsfield Village, 

Herbach, Leon H., A.B. (Brooklyn) Sub. Instr., Dept, of Math., Brooklyn Coll., N. Y., 
1926 64th St., Brooklyn 4* 

Hoeffding, Wassily, Ph.D. (Berlin) 161 West 88 St., New York 24, N. Y. 

Huhndorff, Roland F., B.S. (St. Mary’s Univ.) Ass’t to Ass’t Chief Chemist, The Texas 
Co., Res. Lab., Port Arthur, Texas. 
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James, William C., A.B. (Knox Coll.) Director, Stat. Div., National Safety Council, 20 
N. Wacker Dr., Chicago 6, Ill., 78S5 So. Dobson Ave.^ Chicago 19. 

Lev, Joseph, Ph.D. (Cornell) Ass't Civil Service Examiner, N. Y. C. Civil Service 
Comm., and Lecturer, Teachers College, Columbia Univ., N. Y., 8550 Forest Parkway^ 
Woodhaven 21. 

Linder, Arthur, Ph.D. (Hern) Prof, of applied math, stat.. University of Geneva, Switzer¬ 
land, Avenue de Champel 24-. 

Lord, Frederic M., M.A. (Minnesota) Ass’t Director, Graduate Record Examination, 437 
West 59th St., New York 19, N. Y., 153 W. 6Srd St. 

Marshall, Herbert, B.A. (Toronto) Dominion Stat., Dominion Bureau of Statistics,, 
Ottawa, Canada. 

Meacham, Alan D., Supv., Sorting and Tabulating Station, and Lecturer, School of Bus. 
Adm., Univ. of Mich., Ann Arbor, Mich., 114 Rackham Bldg. 

Miller, Irving, B.S. (C.C.N.Y.) Stat., Bureau of Labor Stat., Washington, D.C.flOOO 
Biltmore St.j N. W., Washington 9. 

Nanda, D., N., M.A. (Agra, India) Graduate student, Univ. of N. Car., Chapel Hill, N. 
Car., Dept, of Statistics. 

Pines, Sylvia F., M.A. (Michigan) Instr., Math, and Stat., 43-17 48th St., Long Island 
City 4, N. Y. 

Quastler, Henry M.D. (Vienna) Medical Radiologist, Carle Hospital Clinic, Urbana, Ill., 
612 W. Nevada. 

Reiersol, Olav, Ph.D. (Stockholm) Teacher of stat., Univ. of Oslo, Oslo, Norway, Interna¬ 
tional House, 600 Riverside Dr., New York 27, N. Y. 

Romanovsky, Vsevolod I., Ph.D. (Moscow) Prof, at the Univ. and Member of the Academy 
of Sciences, Tashkend, U. S. S. R. 

Rust, Charles H., S.J., M.A. (St. Louis) Graduate student, St. Louis Univ., St. Louis, 
Mo., 221 N. Grand Blvd., St. Louis 3. 

Seal, Hilary L., B.Sc. (Univ. Coll., London) Head of Stat. Branch, Room 2, Old Bldg., G., 
Admiralty, Whitehall, London, S. W. 1, England. 

Serhein, Oscar N., Jr., M.S. (Iowa) Graduate student, Columbia Univ., New York 27, 
N. Y., Army Hall, Rm. 333H, 1660 Amsterdam Ave., New York 31. 

Sholl, D., A., B.Sc. (London) Stat. in Math. Stat. Section of Chief Scientific Adviser^s 
Div., Ministry of Works, 81 Lynmouth Ave., Bush Hill Park, Enfield, Middlesex, 
England. 

Siegel, Irving H., M.A. (New York) Chief, Economics Div., Veterans Adm., Washington, 
D. C., 5407 9th St., N. W., Washington 11. 

Sitgreaves, Rosedith, M.A. (Geo. Washington) Ass’t Stat., U. S. Public Health Service 
(on leave); Graduate student, Columbia Univ., New York 27, N. Y., Johnson Hall, 
411 W. 116th St. 

Tama. Joseph. B.A. (Washington) Pfc. U. S. Army, 5250 TIC; GHQ AFPAC; APO 500, 
c/o Postmaster, San Francisco, Calif. 

Tate, Merle W., PM.M. (Harvard), M.A. (Montana) Assoc. Prof, of Educ., Hamilton 
Coll., Clinton, N. Y. 

Thrall, Robert M., Ph.D. (Illinois) Ass’t Prof, of Math., Univ. of Mich., Ann Arbor, 
Mich., 963 Spring St. 

Vaughn, Kenneth W., Ph.D. (Iowa) Director, Graduate Record Examination Office of the 
Carnegie Foundation for the Advancement of Teaching; and. Assoc. Director of Co¬ 
operative Test Service of Amer. Council on Educ., 437 West 59th St., New York 19, N. Y. 

Wallace, Clifford A., Sup’t of Quality, Camera Works, Eastman Kodak Co., 333 State St., 
Rochester, N. Y. 

Wilkins, J., Ernest, Jr., Ph.D. (Chicago) Mathematician, American Optical Co., S. I. D., 
Box A, Buffalo 15, N. Y. 

Wilkinson, Roger I., B.S.E.E. (Iowa State) Member Technical Staff, Bell Telephone Labs., 
463 West St., New York, N. Y’’. 



REPORT ON THE BOSTON MEETING OF THE INSTITUTE 


The twenty-fourth meeting of the Institute of Mathematical Statistics was held 
at the Hotel Statler, Boston, Massachusetts, on Saturday, December 28, 1946. 
The meeting was held in conjunction with the One Hundred Thirteenth Annual 
Meeting of the American Association for the Advancement of Science. The 
following 45 members of the Institute attended the meeting: 

K. J. Arnold, M. S. Bartlett, W. D. Baten, C. I. Bliss, G. W. Brier, G. W. Brown, T. H. 
Brown, B. H. Camp, C. W. Churchman, W. G. Cochran, J. H. Curtiss, D. B. DeLury, P. V. 
Dorweiler, Churchill Eisenhart, Benjamin Epstein, 11. A. Freeman, Hilda Geiringer, II. H. 
Germond, J. A. Greenwood, Boyd Harshbarger, W. A. Hendricks, E. H. C. Hildebrandt, 
W. C. Jacob, H. B. Kaitz, L. F. Knudsen, Walter Leighton, A. J. Lotka, J. W. Mauchly, 
Margaret Merrell, E. B. Mode, Frederick Mosteller, C. M. Mottley, Doris Newman, R. H. 
Noel, H. W. Norton, Otis Pope, C. J. Rees, C. F. Roos, P. J. Rulon, J. W. Tukey, W. M. 
llpholt, F. M. Wadley, C. L. Weaver, C. P. Winsor, W. J. Youden. 

At the morning session, a joint session with the Biometrics Section of the 
American Statistical Association, the following program was presented with 
Professor E. B. Wilson of Harvard University as chairman: 

Topic: The Analysis of Variance in Biology 

Papers: The Assumptions Underlying the Analysis of Variance 

Professor Churchill Eisenhart, University of Wisconsin and The National 
Bureau of Standards 

Some Consequences when the Assumptions are not Satisfied 
Professor W. G. Cochran, North Carolina State College 
The Use of Transformations 

Professor M. S. Bartlett, Cambridge University and the University of 
North Carolina 

Discussion: Professor Boyd Harshbarger, Virginia Polytechnic Institute 
Dr. W. C. Jacob, Long Island Vegetable Research Farm 
Professor C. P. Winsor, Johns Hopkins University 
Dr. W. J. Youden, Boyce Thompson Institute 

The program for the afternoon session, also a joint session with the Biometrics 
Section, under the chairmanship of Dr. E. J. DeBeer, Wellcome Research 
Laboratories, was as follows: 

Topic: The Analysis of Variance in Biology {continued) 

Papers: The Analysis of Covariance 

Professor D. B. DeLury, Virginia Polytechnic Institute 
Discriminant Functions 

Professor George W. Brown, Iowa State College 

Discussion: Professor W. D. Baten, Michigan State College 
Professor C. I. Bliss, Yale University 
Mr. W. A. Hendricks, U. S. Department of Agriculture 
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P. S. DwyeRj 
Secretary. 



ANNUAL REPORT OF THE PRESmENT OF THE INSTITUTE FOR 1946 

New Opportunities 

The return to peacetime conditions presents the Institute with new oppor¬ 
tunities for expanding its activities and usefulness. An increased appreciation 
for mathematical statistics has followed the many contributions made by our 
members to the war effort. The numerous societies interested in specific appli¬ 
cations of statistics have come to look to the Institute both for leadership in 
theory and for playing its part in the dissemination of new results. As a result of 
the drastic interruption in the normal training of students during the war, there 
is unusually keen competition for the services of capable statisticians. Those of 
our members who are engaged in teaching arc responsible for the execution of a 
vigorous training program to meet current and future demands promptly and 
without sacrifice of quality. In short, we are in a position, as never before, to 
advance the development and eflScicnt use of mathematical statistics. The fol¬ 
lowing account of some of our activities during the year will indicate, I believe, 
that the record is creditable. Yet in many instances what has been accomplished 
is only a beginning. 


Meetings 

The Development Committee has repeatedly stressed the desirability of an 
extension in our customary schedule of meetings in order to provide additional 
contacts between mathematical statisticians and the users of statistics. Owing 
to the greater availability of railway and hotel accommodation in 1946, we ob¬ 
tained our first opportunity to put this extension into effect. The regular winter 
meeting with the American Statistical Association and other social science or¬ 
ganizations was resumed at Cleveland in January, while the late summer meeting 
with the mathematicians took place at Cornell in September. In addition, two 
meetings were held with different sections of the American Association for the 
Advancement of Science, at St. Louis in March and at Boston in December. 
On both occasions the programs were expository and attracted large audiences. 
Finally, at the invitation of Princeton University, a one-day meeting at Princeton 
in November was devoted to the analysis of variance. While no joint sessions 
were conducted with engineering or industrial societies, several of our members 
took prominent parts in the programs of such societies. 

For the near future, it seems desirable to continue the practice of meeting in 
the winter with the ASA and social science groups and in the summer with the 
mathematical groups. In 1947 these meetings will be at Atlantic City, January 
24-27 and at Yale, September 1-5 respectively. It is not known whether con¬ 
ditions in future years will produce a return to Christmas rather than January 
meetings: for the present the hotel situation swings the balance in favor of 
January. 
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In 1946 the membership of the program committee was enlarged so that it 
would be better equipped to arrange joint meetings with other societies. We 
owe our thanks to the members for their successful efforts in the face of difficulties 
which still attend the planning of a meeting. 

Annals 

Despite the scarcity of manuscripts in the later stages of the war, our editor, 
Professor S. S. Wilks, succeeded throughout in maintaining the annual volumes 
of the Annals at their usual size. During 1946, scarcity gave way to plenty. 
The number of papers of good (quality submitted in recent months is sufficiently 
great that there will be more than enough, by current estimates, to fill the 1947 
volume. To narrow the scope of the Annals or to reject good papers would be 
undesirable. Accordingly, thb Directors have authorized an increase of 100 
pages in the 1947 volume if this is necessary to insure the publication of all ac¬ 
ceptable papers. 

A gratifying testimony to the prominence of the Annals in its field is the marked 
increase in the demand for back numbers. Our Secretary-Treasurer reports 
that sales amounted to sS3,235. To meet actual or anticipated orders, eleven 
issues were reprinted during 1946 at a cost of $2,809. 

For most members of the Institute, even those who serve on the Board, work 
on Institute affairs occupies only a minor portion of our time. The editor is 
never free from some forthcoming publication deadline. Initial perusal of manu¬ 
scripts, selection of referees, editorial decisions, handling of the production phases 
of publication and much miscellaneous correspondence (not all of it pleasant) 
make editorial work a daily preoccupation, year in and year out. An annual 
word of thanks is an inadequate expression of our indebtedness to Professor 
Wilks. 


Membership and Finance 

At the beginning of 1945 there were 606 members. A year later this figure 
had increased to 777 and at the end of 1946 the figure stood at 900. A fifty per¬ 
cent increase in two years is another evidence of the healthy growth of the Insti¬ 
tute. It has been attained to a considerable extent through the hard work of 
our Secretary-Treasurer, P. S. Dwyer and the cooperation of individual members. 

The Secretary-Treasurer also reports a very satisfactory net gain in assets 
of $2,627 during the year. Nevertheless, financial problems may arise in the 
near future. Printing and other costs have risen sharply, and the printing of an 
enlarged Annals will be an additional drain on our resources. Both the Member¬ 
ship and Development Committees have given some thought to the need for 
additional revenue that may face us soon. They have recommended considera¬ 
tion of the possibility of Institutional Memberships, a device that has been found 
satisfactory by some other societies. A continued growth in membership will 
also help greatly to finance expanded activities. 
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Committees 

Inter-society affairs: The report of the 1944 Committee on Development, stress¬ 
ing the need for closer cooperation amongst the various societies interested in 
statistics, provided the stimulus for active efforts in this direction. A meeting 
of representatives of these societies was called early in 1945 at the invitation of 
the American Statistical Association. This meeting suggested that a reconsti¬ 
tution of the ASA might enable it to become the central binding organization. 
Accordingly, a committee of the ASA has worked for a considerable time on a 
revision of the ASA constitution, which it is intended to submit to the votes of 
ASA members early in 1947. The new constitution provides for representation 
from other societies on the Council of the ASA, should these societies decicie to 
associate or affiliate Avith the ASA. . 

From our own point of view, it has seemed wise to delay action on certain 
internal affairs Avhile awaiting the outcome of these developments in the ASA. 
Thus a statement of policy with regard to the formation of chapters of the IMS 
is needed and the problem has been considered both by a special committee in 
1945 and by the Development Committee in 1946. The latter committee recom¬ 
mends that no decision be made pending examination of the provisions for joint 
sponsorship of local and regional chapters in the new ASA constitution. Simi¬ 
larly, our own Committee on Revising the Constitution and By-Laws has de¬ 
ferred a final report until the attitude of our members towards the new develop¬ 
ments can be expressed. It is to be hoped that decisions can be taken in 1947. 

Tabulation: The advances made in recent years in the construction of new’ types 
of computing equipment justified an enlargement of our Committee on Tabula¬ 
tion, which now includes experts both on the building of machines and on the 
calculation and use of tables. The committee plans to keep our members in¬ 
formed of progress in this field. 

Government Service: Dr, W. Edwards Doming served as chairman of a new com¬ 
mittee on Mathematical Statistics and Statisticians in the Government Seiwice. 
Although the federal government employs many mathematical statisticians, 
explicit recognition of the profession is lacking in many instances. As has hap¬ 
pened in other fields, statisticians are sometimes officially classed as economists 
and little provision is made for mathematical statisticians in recruitment policies. 
Moreover, it is probable that a number of branches of the government, at present 
unaware of the functions of a statistician, could employ several with profit. 
The neAv committee will endeavor to insure that mathematical statistics is recog¬ 
nized and effectively utilized in the federal service. 

Assistance to libraries: Like other professional societies, the Institute has re¬ 
ceived a number of appeals from libraries in war areas whose periodicals were 
looted or destroyed during the war. After careful consideration, the Board 
decided that official action should be limited to the free provision of missing 
copies of the Annals to all former subscribers who intend to renew subscriptions 
for the future. In addition, a committee A\ith Professor J. Neyman as chairman 
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was appointed to establish a procedure by which gifts of individual members 
(books, reprints, back numbers of the Annals or cash for the purchase of back 
numbers) could be handled. At the suggestion of this committee a general 
appeal for the small sum of 50 cents per member was circulated with the Decem¬ 
ber billing. Individual collections are also being made at certain centers. 

Teaching: The Committee on Teaching has not made as much progress as it 
would have liked, owing to the dispersal of its members and the taking up of new 
civilian posts. Members have, however, cooperated with the Committee on 
Applied Mathematical Statistics of the National Research Council, which is 
engaged on a somewhat similar survey. 

Rietz lecture: The first lecturer in the new series of lectures in honor of the late 
Henry Lewis Rietz will be Professor A. Wald. His topic will be “Sequential 
Estimation and Multi-Decisions^’. The lecture will be delivered in connection 
with the Yale meetings, September 1947. 

Representatives: In addition to its committee work, the Institute cooperates, 
through representatives, with the Division of Physical Sciences of the National 
Research Council, the Joint Committee for the Development of Statistical Ap¬ 
plications in Engineering and Manufacturing, the American Association for the 
Advancement of Science, the Inter-Society Committee on Federation and the 
Policy Committee for Mathematics. The last committee, wdiich was appointed 
in 1946, will consider important problems that affect the mathematics profession 
as a whole. 

Nominatiom: The Committee on Nominations, consisting of Professor P. H. 
Rider (chairman). Professor B. H. Camp and Professor G. M. Cox, has made the 
following nominations for officers in 1947. 

President: W. Feller 
Vice-Presidents: J. H. Curtiss 
M. II. Hansen 

Secretary-Treasurer: P. S. Dwyer 

While it is perhaps improper to comment on nominations, I should like to 
express my personal appreciation of Professor Dwyer’s action in being willing 
to offer himself for re-nomination as Secretary-Treasurer. The successful opera¬ 
tion of the Institute rests mainly on the Secretary-Treasurer, and the demands 
of the Office are even more continuous and exacting than those on the editor. 
Professor Dwyer’s splendid work during his first three years of office, carried on 
at considerable sacrifice of his research interests, deserves thi- f)est thanks and 
appreciation of every member. 

In conclusion, it is a pleasure to express my sincerest thanks to all committee 
chairmen and members and to all representatives for their excellent work for the 
good of the Institute, and to all Institute members for their loyal support. 

W. G. CO( HIIAV, 
Rrcsulcnty 1946. 
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Committee 

Development 

Committees of the Institute 

Personnel 

E. G. Olds (chairman), C. I. Bliss, M. A. 
Girshick, F. C. Mosteller, P. S. Olmstead, 
H, Scheff^. 

Membership 


W. Feller (chairman), C. C. Craig, P. A. 
Horst, T. Koopmans 

Program 


J. H. Curtiss (chairman), M. Friedman, B. 
Harshbarger, W. N. Hurwitz, A. M. Mood, 
F. C. Mosteller, J. W. Tukey 

Mathematical Statistics and 
Statisticians in the Govern¬ 
ment Service 

W. E. Deming (chairman) 

Revising the Constitution- 
and By-Laws 

M. H. Hansen (chairman), C. I. Bliss, A. T. 
Craig, J. H. Curtiss, W. Shewhart 

Tabulation 


C. Eisenhart (chairman), P. S. Dwyer, H. 
Goldstine, A. N. Lowan, H. W. Norton, G. R, 
Stibitz 

Teaching 


H. Hotelling (chairman), W. Bartky, W. E. 
Deming, M. Friedman 

Nominations 


P. R. Rider (chairman), B. H. Camp, G. M. 
Cox 

Finance 


P. S. Dwyer (chairman), L. A. Knowler, C. F, 
Roos, F. F. Stephan 

Subscription to Purchase An¬ 
nals for Countries Devas¬ 
tated by War 

J. Neyman (chairman), W. Feller, P. L. Hsu 

Society 

Inter-Society Committee 
Federation 

on 

Representatives 

J. H. Curtiss, P. S. Olmstead 

Policy Committee for Mathe¬ 
matics 

W. Feller 
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Society 

Joint Committee for the De¬ 
velopment of Statistical 
Applications in Engineer¬ 
ing and Manufacturing 

American Association for the 
Advancement of Science 

Division of Physical Sciences, 
NRC 


Repreeentativea 

F. C. Mosteller, S. S. WUks 

G. W. Snedecor 

W. Bartky 



REPORT OF THE SECRETARY-TREASURER OF 
THE INSTITUTE FOR 1946 

The Institute of Mathematical Statistics held five meetings during 1946, at 
Cleveland on January 24-27, at St. Louis on March 30, at Ithaca on August 
22-23, at Princteon on November 1, and at Boston on December 28. 

The large number of meetings has necessitated frequent mailings to the 
membership. Memoranda to members, with appropriate enclosures, were sent 
out in January, March, June, July, October, and November. 

The Secretary-Treasurer wishes to acknowledge the cooperation of the mem¬ 
bers of the Institute in paying bills promptly, in considerable activity leading to 
an increase in membership, and in general looking after the interests of the In¬ 
stitute. 

At the beginning of 1946 the Institute had 777 members. During the year 
180 new members joined the Institute, an increase of 23%. However, during 
1946 the Institute lost 57 members. Of these, 15 resigned, 37 were dropped for 
non-payment of dues, and 5 are deceased. Some of the 37 dropped we have 
been unable to contact, and it is very probable that, in some cases, membership 
will be resumed in the future. The net increase in members during the year was 
123, or about 16%, making a total of 900 members. 

The following members died during the year: 

Professor 0. F. Banos 
Professor S. A. Cudmore 
Professor Dunham Jackson 
Dr. Walter F. Schilling 
Professor C. C. Wagner 

The office of the Secretary-Treasurer sent a reprint of an Annals article and 
information about the Institute to 1800 persons interested in Quality Control. 
At least 28 of the new members became members as a result of this drive. As a 
continuation of a campaign started in 1945, the Institute also sent literature 
about the Annals to several hundred libraries and laboratories. 

The Secretary-Treasurer wishes to acknowledge the continued assistance of 
Professor Lloyd Knowlcr in caring for the back issues of the Annals which are 
stored at Iowa City. 

A few comments about the financial statement which appears below are in 
order. In addition to the increase in membership, mentioned above, the chief 
rise in income resulted from the unprecedented sales in back issues which 
amounted to $3,234.88, an increase over the preceding year (the previous high) 
of 86%. These heavy sales, however, depleted the supplies of many of our early 
issues, so that we were forced to reprint eleven of these issues and also the cumula¬ 
tive index during the year. This cost $2,809.00 (for 500 copies of each) and in¬ 
dicates that a much larger portion of our assets is in inventory, as shown in Ex¬ 
hibit D 
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Following the instructions of the Finance Committee, Professor H. C. Carver 
was paid for his share of all issues in which he and the Institute had joint owner¬ 
ship. 

Nine members have paid life memberships during the year, increasing the 
total of life membership funds by $812.50. 

The net gain in assets of $2,627.23 is very satisfactory even though this gain 
is evident in increased inventory and not in a better cash position. 


FINANCIAL STATEMENT 
December 31, 1945, to December 31, 1946 
A Receipts 


Balance on Hand, December 31, 1945 

$7,548 

22 

Dues 

4,638. 

.40 

LiifE Membership Paymln rs 

812 

50 

Subscriptions 

2,057 

54 

Sale of Back Numbers 

3,234 

88 

Income from Investments 

150 

00 

Miscellaneous 

121 

29 

Toial 

$18,562 

83 


II LxPENDnURfS 

Annals—C uurlni 

Office of Editor $125 00 

Waverly Press 4,666 27 


$4,691 27 

Annais— Back iSUMBLRb 

Purchase from H C. Caiver 644 50 

Reprinted 500 copies 2,809 00 

Voi I fni, II r2, II III $6, IV ^jfi, VII ^3, VII $ 2 , viii 
/l^l, 2, 3, 4, Cumulative Index 

Iowa City Office 41 46 

Binding 68 00 


Ofiice of President 

Mathematical Review's 

OiFiCE OF thi Secretary-Treasurer 

Printing, Mimeographing, programs, etc (including stamped 


envelopes) $967 14 

Printing 1800 copies of Wald-Wolfowitz article 140 00 

Postage and supplies 375 00 

Clerical help 1,420 25 


3,562 96 
25 62 
100 00 


2,902.39 

Miscellaneous 39 04 

Balance on Hand, December 31, 1946 (Cash and Bonds) 7,241.55 


$18,562 83 
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C. Summary of Receipts and Expenditures 

Balance on Hand,* December 31, 1945 $7,548.22 

Receipis during 1946 11,014.61 

Expenditlres during 1946 11,321.28 

Balance on Hand,* December 31, 1946 7,241.55 

D. Comparison op Assets on December 31, 1945 and December 31, 1946 

i9J^ 1946 


US Government G Bonds 

$6,000 00 

$5,000 00 

Life Membership Funds 

/ 888 00 
\ 327.00 

f 1888.00 Bonds 
\ 139 50 Bank Dep. 

Additional Bank Deposits 

333.22 

214.05 

Current Accounts Receivable 

255 35 

452 62 

Estimated Value (Cost) of back issues of Annals 

. 4,497.95 

7,234 58** 

Total 

$12,301.52 

14,928.75 

Net Gain 1946 


2,627.23 


E. Liabilities of Institute of Mathematical Statistics as op December 31,1946 

All bills which have been presented have been paid and there are no outstanding accounts 
against the Institute The $2027.50 in Life Membership payments require the Institute to 
provide the privileges of membership for life for the 26 members who have made payments. 
Also, $2686 71 should be credited to 1947 dues and subscriptions 


December 31, 1946 


Paul S. Dwyer 
Secretary^ Treds urer. 


* In form of bank deposit and government bonds. 

** Value of Annals calculated at 67 cents per copy, and based on physical inventoryl 



AHKTJAL TJEPOUT 0¥ THE EHTtOE EOE 1946 

During 1946 there was a considerable increase in the number of manuscripts 
submitted to the Editorial Committee of the Annals, A total of 49 papers in¬ 
cluding 18 short notes were published in the 1946 volume of the Annals, The 
publication of these papers together with various official reports of the Institute 
and the Directory of the Institute required a total of 555 pages. Plans are al¬ 
ready under way to expand the 1947 volume of the Annals to 600 pages. 

During recent years there has been a very noticeable broadening of interest 
in the field of probability and statistical theory on the part of readers and con¬ 
tributors to the Annals, Contributors to the 1946 volume came from university 
departments of astronomy, biology, mathematics, sociology and statistics; from 
Army, Navy and other government groups; and from industrial laboratories and 
quality control departments. More recently, contributions have been received 
from physicists, chemists and other groups. More contributions are now being 
received from overseas than in previous years. Every effort is being made to 
keep the Annals balanced with respect to these various directions of interest in 
probability and statistical problems. It is believed that one of the most effec¬ 
tive things which could be done for the readers of the Annals is to publish ex¬ 
pository articles from time to time on new fields of development in probability 
and statistical theory. Invitations have been accepted by several individuals to 
prepare such articles. 

Dr. Thornton C. Fry has asked to be relieved from the Editorial Com¬ 
mittee, as of January 1, 1947. The Editor wishes to take this opportunity to 
express his gratitude for the service which Dr. Fry has rendered in connection with 
the editorial work on the Annals during the past nine years. 

On behalf of the Ekiitorial Committee for the Annals, the Editor wishes to 
acknowledge with thanks the refereeing assistance which has been provided by 
the following persons during 1946: R. L. Anderson, T. W. Anderson, David 
Blackwell, Z. W. Birnbaum, K. L. Chung, W. J. Dixon, J. L. Doob, M. A. 
Girshick, T. E. Harris, L. Ilenkin, M. Kac, Irving Kaplansky, Bradford F. Kim¬ 
ball, T. Koopmans, H. Levenc, H. B. Mann, P. J. McCarthy, F. C. Mosteller, H. 
E. ;^bbins, D. F. Votaw, J. E. Walsh and C. P. Winsor. The Editor is also 
indebted to the following individuals at Princeton University for preparation of 
manuscripts for the printer, and other editorial assistance: Mrs. Gladys B. Ruling, 
Mrs. Eleanor C. Schoenly and J. E. Walsh. 

S. S. Wilks 
Editor, 

December 31, 1947 
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CONSTITUTION 

OF THE 

mSTITDTE OF MATHEMATICAL STATISTICS 


ARTICLE I 
Name and Purpose 

1. This organization shall be known as the Institute of Mathematical Statistics. 

2. Its object shall be to promote the interests of mathematical statistics. 

ARTICLE II 

Membership 

1. The membership of the Institute shall consist of Members, Fellows, Honorary 
Members, and Sustaining Members. 

2. Voting members of the Institute shall be (a) the Fellows, and (b) all others. Junior 
members excepted, who have been members for twenty-three months prior to the date 
of voting. 

3. No person shall be a Junior Member of the Institute for more than a limited term as 
determined by the Committee on Membership and approved by the Board of Directors. 

ARTICLE III 

OpncERs, Board of Directors, and Committee on Membership 

♦ 

1. The Officers of the Institute shall be a President, two Vice-Presidents, and a Secre¬ 
tary-Treasurer. The terms of office of the President and Vice-Presidents shall be one 
year and that of the Secretary-Treasurer three years. Elections shall be by majority 
ballots at Annual Meetings of the Institute. Voting may be in person or by mail. 

(a) Exception. The first group of Officers shall be elected by a majority vote of the 
individuals present at the organization meeting, and shall serve until December 31,1936. 

2. The Board of Directors of the Institute shall consist of the Officers, the two previous 
Presidents, and the Editor of the Official Journal of the Institute. 

3. The Institute shall have a Committee on Membership composed of a Chairman and 
three Fellows. At their first meeting subsequent to the adoption of this Constitution, the 
Board of Directors shall elect three members as Fellows to serve as the Committee on 
Membership, one member of the Committee for a term of one year, another for a term of 
two years, and another for a term of three years. Thereafter the Board of Directors shall 
elect from among the Fellows one member annually at their first meeting after their elec¬ 
tion for a term of three years. The president shall designate one of the Vice-Presidents as 
Chairman of this Committee. 


ARTICLE IV 
Meetings 

1. A meeting for the presentation and discussion of papers, for the election of Officers, 
and for the transaction of other business of the Institute shall be held annually at such 
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time as the Board of Directors may designate. Additional meetings may be called from 
time to time by the Board of Directors and shall be called at any time by the President 
upon written request from ten Fellows. Notice of the time and place of meeting shall be 
given to the membership by the Secretary-Treasurer at least thirty days prior to the 
date set for the meeting. All meetings except executive sessions shall be open to the 
public. Only papers accepted by a Program Committee appointed by the President may 
be presented to the Institute. 

2. The Board of Directors shall hold a meeting immediately after their election and 
again immediately before the expiration of their term. Other meetings of the Board 
may be held from time to time at the call of the President or any two members of the 
Board. Notice of each meeting of the Board, other than the two regular meetings, 
together with a statement of the business to be brought before the meeting, must be 
given to the members of the Board by the Secretary-Treasurer at least five days prior to 
the date set therefor. Should other business be passed upon, any member of the Board 
shall have the right to reopen the question at the next meeting. 

3. Meetings of the Committee on Membership may be held from time to time at the call 
of the Chairman or any member of the Committee provided notice of such call and the 
purpose of the meeting is given to the members of the Committee by the Secretary- 
Treasurer at least five days before the date set therefor. Should other business be passed 
upon, any member of the Committee shall have the right to reopen the question at the 
next meeting. Committee business may also be transacted by correspondence if that 
seems preferable. 

4. At a regularly convened meeting of the Board of Directors, four members shall 
constitute a quorum. At a regularly convened meeting of the Committee on Member¬ 
ship, two members shall constitute a quorum. 

ARTICLE V 

PUBLICATIONB 

1. The Annals of Mathematical Statistics shall be the Official Journal for the Institute. 
The Editor of the Annals of Mathematical Statistics shall be a Fellow appointed by the 
Board of Directors of the Institute. The term of office of the Editor may be terminated 
at the discretion of the Board of Directors. 

2. Other publications may be originated by the Board of Birectors as occasion arises. 

ARTICLE VI 
Expulsion or Suspension 

1. Except for non-payment of dues, no one shall be expelled or suspended except by 
action of the Board of Directors with not more than one negative vote. 

ARTICLE VII 

Amendments 

1. This constitution may be amended by an affirmative two-thirds vote at any regu¬ 
larly convened meeting of the Institute provided notice of such proposed amendment 
shall have been sent to each voting member by the Secretary-Treasurer at least thirty 
days before the date of the meeting at which the proposal is to be acted upon. Voting 
may be in person or by mail. 
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ARTICLE I 

Duties op the Officers, the Editor, Board op Directors, and 
Committee on Membership 

1. The President, or in his absence, one of the Vice-Presidents, or in the absence of the 
President and both Vice-Presidents, a Fellow selected by vote of the Fellows present, 
shall preside at the meetings of the Institute and of the Board of Directors. At meetings 
of the Institute, the presiding officer shall vote only in the case of a tie, but at meetings 
of the Board of Directors he may vote in all cases. At least three months before the date 
of the annual meeting, the President shall appoint a Nominating Committee of three 
members. It shall be the duty of the Nominating Committee to make nominations for 
Officers to be elected at the annual meeting and the Secretary-Treasurer shall notify all 
voting members at least thirty days before the annual meeting. Additional nomina¬ 
tions may be submitted in writing, if signed by at least ten Fellows of the Institute, up to 
the time of the meeting. 

2. The Secretary-Treasurer shall keep a full and accurate record of the proceedings 
at the meetings of the Institute and of the Board of Directors, send out calls for said 
meetings and, with the approval of the President and the Board, carry on the corre¬ 
spondence of the Institute. Subject to the direction of the Board, he shall have charge 
of the archives and other tangible and intangible property of the Institute and once a year 
he shall publish in the Annals of Mathematical Statistics a classified list of all Members and 
Fellows of the Institute. He shall send out calls for annual dues and acknowledge receipt 
of same; pay all bills approved by the President for expenditures authorized by the Board 
or the Institute; keep a detailed account of all receipts and expenditures, prepare a finan¬ 
cial statement at the end of each year and present an abstract of the same at the annual 
meeting of the Institute after it has been audited by a Member or Fellow of the Institute 
appointed by the President as Auditor. The Auditor shall report to the President. 

3. Subject to the direction of the Board, the Editor shall be charged with the responsi¬ 
bility for all editorial matters concerning the editing of the Annals of Mathematical Sta¬ 
tistics. He shall, with the advice and consent of the Board, appoint an Editorial Commit¬ 
tee of not less than twelve ^embers to co-operate with him; four for a period of five years, 
four for a period of three years, and the remaining members for a period of two years, ap¬ 
pointments to be made annually as needed. All appointments to the Editorial Com¬ 
mittee shall terminate with the appointment of a new Editor. The Editor shall serve as 
editorial adviser in the publication of all scientific monographs and pamphlets authorized 
by the Board. 

4. The Board of Directors shall have charge of the funds and of the affairs of the 
Institute, with the exception of those affairs specifically assigned to the President or to 
the Committee on Membership. The Board shall have authority to fill all vacancies 
ad interim, occurring among the Officers, Board of Directors, or in any of the Committees. 
The Board may appoint such other committees as may be required from time to time 
to carry on the affairs of the Institute. The power of election to the different grades of 
Membership, except the grades of Member and Junior Member, shall reside in the Board. 

5. The Committee on Membership shall prepare and make available through the 
Secretary-Treasurer an announcement indicating the qualifications requisite for the 
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different grades oi membership. The Ckjmmittee shall review these qualiffcations period¬ 
ically and shall make such clianges in these quahffcations and make such recommendations 
with reference to the number of grades of membership as it deems advisable. The power 
to elect worthy applicants to the grades of Member and Junior Member shall reside in the 
Committee, which may delegate this power to the Secretary-Treasurer, subject to such 
reservations as the Committee considers appropriate. The Committee shall make recom¬ 
mendations to the Board of Directors with reference to placing members in other grades 
of membership. The Committee shall give its attention to the question of increasing the 
number of applicants for membership and shall advise the Secretary-Treasurer on plans 
for that purpose. 


ARTICLE II 
Dues 

1. Members shall pay five dollars at the time of admission to membership and shall 
receive the full current volume of the Official Journal. Thereafter, Members shall pay 
five dollars annual dues. The annual dues of Junior Members shall be two dollars and 
fifty cents. 

The annual dues of Fellows shall be five dollars. The annual dues of Sustaining 
Members shall be fifty dollars. Honorary Members shall be exempt from all dues. 

(a) Exception. In the case that two Members of the Intitute are husband and wife 
and they elect to receive between them only one copy of the Official Journal, the annual 
dues of each shall be three dollars and seventy-five cents. 

(b) Exception. Any Member or Fellow may make a single payment which will be 
accepted by the Institute in place of all succeeding yearly dues and which will not other¬ 
wise alter his status as a Member or Fellow. The amount of this payment will depend 
upon the age of this Member or Fellow and will be based upon a suitable table and rate of 
interest, to be specified by the Board of Directors. 

(c) Exception. Any Member or Junior Member of the Institute serving, except as a 
commissioned officer, in the Armed Forces of the United States or of one of its allies, may 
upon notification to the Secretary-Treasurer be excused from the payment of dues until the 
January first following liis discharge from the Service. He shall have all privileges of 
membership except that he shall not receive the Official Journal. However during the 
first year of his resumed regular membership he may have the right to purchase, at $2.50 
per volume, one copy of each volume of the Official Journal published during the period 
of his service membership. 

2. Annual dues shall be payable on the first day of January of each year. 

3. The annual dues of a Fellow, Member, or Junior Member include a subscription to 
the Official Journal. The annual dues of a Sustaining Member include two subscriptions 
to the Official Journal. 

4. It shall be the duty of the Secretary-Treasurer to notify by mail anyone whose dues 
may be six months in arrears, and to accompany such notice by a copy of this Article. 
If such person fail to pay such dues within three months from the date of mailing such 
notice, the Secretary-Treasurer shall report the delinquent one to the Board of Directors, 
by whom the personas name may be stricken from the rolls and all privileges of member¬ 
ship withdrawn. Such person may, however, be re-instated by the Board of Directors 
upon payment of the arrears of dues. 
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ARTICLE III 
Salakies 

1. The Institute shall not pay a salary to any Officer, Director, or member of any 
committee. 


ARTICLE IV 
Amendments 

1, These By-Laws may be amended in the same manner as the Constitution or by a 
majority vote at any regularly convened meeting of the Institute, if the proposed amend¬ 
ment has been previously approved by the Board of Directors. 



PROBLEMS IN PROBABILITY THEORY 

By Harald Cramer 
University of Stockholm 

1. Introduction. The following survey of problems in probability theory 
has been written for the occasion of the Princeton Bicentennial Conference on 
‘‘The Problems of Mathematics,’* Dec. 17-19, 1946. It is strictly confined to 
the purely mathematical aspects of the subject. Thus all questions poncerned 
with the philosophical foundations of mathematical probability, .frith its 
ever increasing fields of application, will be entirely left out. 

No attempt to completeness has been made, and the choice of the problems 
considered is, of course, highly subjective. It is also necessary to i:)oint out 
explicitly that the literature of the war years has only recently—and still far 
from completely—been available in Sweden. Ouing to this fact, it is almost 
unavoidable that this paper will be found incomplete in many respects. • 

1. FUNDAMENTAL NOTIONS 

2, Probability distributions. From a purely matHjhatical pdint of view, 
probability theory may be regarded as the theory of certain classes of additive 
set functions^ defined on spaces of more or less general types. The basic struc¬ 
ture of the theory has been set out in a clear and (*oncise way in the well-known 
treatise by Kolmogoroff [53]. We shall begin by recalling some of the main 
definitions. . Note that the word additive^ when used in connection with sets 
or set functions, uill always refer to a finite or enumerable sequence of sets. 

Ix 3 t w denote a variable point in an entirely arbitrary space B, and consider 
an additive class U of sets in such thal the uhole space Q itself is a member of 
r. Further, let PiS) be an additive set function, defined for all sets belonging 
to the class U, and suppose that 

P(S) ^ 0 for all S in T, 

• P(12) = 1. 

We shall then say that P{S) is a probability measure, which defines a probability 
distribution in For any set S in (\ the quantity PiS) is called the probability 
of the event expressed by (he relation co Cl >Sf, i.e. the event that the variable 
point o) takes a value belonging to S. Accordingly we ^vrite 

PiS) = P(a) C S). 

Suppose now that co' = giia) is a function of the variable point ca, defined 
throughout ^^e space fl, the values w' being points of another arbitrary space 
B'. Let S^ be a set in ft' and denote by S the set of all points w such that w' = 
gi<ji) belongs to S', Whenever S belongs to C, we define a set function P'iS') 
by writing 

P'(5') = PiS). 
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It is then easy to see that P'(S') is defined for all S' belonging to a certain 
additive class C' in the new space and that P'(S') is a probability measure 
in Q', such that P'(S') signifies the probability of the event w' C S' (which is 
equivalent to w C S). We shall say that P'(S') is attached to the probability 
distribution in Q' which is induced by the given distribution in Q and the function 

w' = gM- 

3. Random variables. Consider in particular the case when oj' is a real 
number such that f = g(o)) is a real-valued C-measurable function of the 
argument to. Then C' includes the class Bi of all Borel sets S' of the space = 
Ri of all real numbers, and avc shall call { a one-dimensional real random variable. 
The probability of the event { C /S' is uniquely defined for any Borel set S' of 
Ri , as soon as the function 

F(x) = P(i ^ x) 

is known for all real x. F(x) is called the distribution function (d.f.) of the 
random variable f. If the function f = g{u)) is integrable over U with respect 
to the measure P(S), we write 

if = f g(o)) dP = f xdF(x), 

Jq J-oo 

and denote this cxpi'ession as the expectation or mean value of the random vari¬ 
able f. Any real-valued B-measurable function rj = h(^) is also a random 
variable with the probability distribution induced by the original co-distribution 
and the function v = h(g(cj)). If rj is integrable over H ^^ith respect to P, its 
mean value may be written in the fonn 

Eri = Ehii) = f h(g(a))) dP = C h{x) dF(x). 

Jq %/-oo 

More generally, if w' = (^i, • • • , f„) is a point in an /^-dimensional Euclidean 
space Rn , while C' includes the class P„ of all Borel sets of P„ , we are con¬ 
cerned with an n-dimensional real random variable. The distribution of this 
variable, which is also called the joint distribution of the n one-dimensional 
variables , • • • , , is uniquely defined, as soon as the joint d.f. 

F{Xi , • • • , aJn) = P(fl ^ , • • • , fn ^ Xn) 

is known for all real a*i, • • • , a:n . 

The variables , •••,$» are said to be independent, if F{xx , • • • , iCn) = Pi(«i) 

• • • P«(rc»), where F„{Xy) is the d.f. of the variable . 

The extension to complex random variables is obvious. Suppose e.g. that 
f = g{<^) and V = A(a)) are two one-dimensional real variables, and consider 
the complex variable f -b + f/i(co). By definition, we identify the 

distribution of this variable with that of the two-dimensional real variable 
(f, ri), and we put 


"b ^* 7 ) “ *b iEri. 
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Joint distributions of several complex variables are introduced in a correspond¬ 
ing way. 

4. Characteristic functions. If { is a one-dimensional real random variable, 
the mean value 

^{z) = = r e*"dF(x) 

J —00 

exists for all real s, and we have 

I ^(z) I ^ 1, ^(0) = 1. 

<p(z) is called the characteristic function (c.f.) of the distribution corresponding 
to the variable f. The reciprocal formula (L4vy) 

F(x) - F(y) = - lim / --- <p(z) dz, 

JiTTX Z-*ao J~Z Z 

which holds for any continuity points x and y of F, shows that there is a one- 
one correspondence between the di. F{x) and the c.f. ^( 2 ). As we shall see 
below, the c.f. provides a powerful analytical tool ^r operations with prob¬ 
ability distributions. 

When a complex-valued function <p{z) of the real variable z is given, it is 
often important to be able to decide whether (p{z) is or is not the c.f. of some 
distribution. If we assume a priori that ^(0) = 1, each of the following condi¬ 
tions is necessary and sufficient for <p{z) to be a c.f. 

• *A . <p{z) should be bounded and continuous for all 2 , and such that the integral 

dz du 

Jo Jq 

is real and non-negative for all real x and all A > 0 (Cram 6 r [ 11 ], in simplifica¬ 
tion of an earlier result due to Bochner, [4]). 

B, There should exist a sequence of functions ^i(z), ^ 2 ( 2 ), • • • such that 

ip{z) = lim / ^n(x + z)^„(x) dx 

n-*oo J —00 

holds uniformly in every finite z-interval (Khintchine, [45]). 

These general theorems arc not always easy to apply in practice. Among 
less general results which are more easily applicable, we mention the almost 
trivial fact that a function (p{z) which near z = 0 is of the form <^(z) = 1 + o(z*) 
cannot be a c.f. unless <p(z) = 1 for all z, and the two following theorems: 

1 ) An integral function <p(z) of order 7 < 1 can never be a c.f. (L^vy, [64]), and 

2 ) an integral function <p(z) of finite order 7 > 2 cannot be a c.f. unless the 
convergence exft^nent of its zeros is equal to 7 (Marcinkiewicz, [72]). The 
latter result shows e.g. that no function of the form where g{z) is a poly¬ 
nomial of degree > 2 , can be a c.f. 

It would be highly desirable to obtain further results in this direction. 
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The c.f. of the joint distribution of n real random variables fi, • • • , f» is the 
function ip{zi ^ ^ Zn) defined by the relation 

<p{zi ,•••,*»)= 

Most of the above results for c.f. in one variable can be directly generalized 
to the multi-variable case. 

5. Random sequences and random functions. Let Z be a variable point in 
an arbitrary space T, and consider the space 12, where each point w is a real¬ 
valued function w = x{t) of the variable argument t. Let h, •••,<» be any 
finite set of distinct points /. The set of all functions w = x{t) satisfying the 
inequalities 

(ij <[ x{tj) ^ bj j (J = 1, * • • , n), 

will be called an interval in the space 12. The Borel sets in 12 will be defined as 
the smallest additive class B of sets in 12 containing all intervals. 

Suppose now that, for any choice of n and the tj , the variables x(ti)y • • •, x{tn) 
are random variables having a knowm n-dimensional joint distribution. If the 
family of all distributions corresponding in this way to finite sequences h , 
-' •, tn satisfies certain obvious consistency conditions, a fundamental theorem 
due to Kolmogoroff asserts that this family determines a unique probability 
distribution in the space 12 of all functions x{t). The corresponding probability 

P{S) = P(x{t) C S) 

is uniquely defined for all Borel sets B of 12. 

Consider in particular the case where T is the set of non-negative integers 
/ = 0, 1, 2, • • • . The space 12 then is the space of all sequences (a*o, Xi, • • •) 
of real numbers. As soon as the joint distribution of any finite number of 
variables x,^ ^ • • • , is defined, and these distributions are mutually con¬ 
sistent, it then follows that there is a unique probability distribution of the 
random sequence Uo, , *••)» tbe corresponding probability being defined 

for every Borel set of the space 12 of sequences. Similarly we may consider the 
doubly infinite sequence (• ■ • , x-i , x©, , • • •). 

Consider further the more general case when T is any set of real numbers. 
Then 12 is the space of all real-valued functions co == x{t) defined on the set T, 
and as before the knowledge of the distributions for all finite sets of variables 
•r(2i), • • • , x{tn) permits us to determine a probability distribution in the space 
12 of random functions x{t)j the probability P(S) = P(x{t) C S) being always 
defined for all Borel sets S in 12. 

The generalization of the above considerations to complex-valued random 
sequences and functions is immediate. 

• 

6 . Various modes of convergence. Consider a sequence Fi{x)^ P^ix), 

of d.f:s, and let the corresponding c.f:s be ^i(0, v? 2 ( 0 , * * • In order that Fn{x) 
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converge to a d.f. F{x)y in every continuity point of the latter, it is necessary 
and sufficient^ that <pn{i) converge for every real / to a limit <p{t) which is con¬ 
tinuous at / = 0. Then (p{t) is the c.f. corresponding to the d.f. F{x). 

Further, let x and Xi ^ x^, • • • be complex-valued random variables, such 
that the random sequence (x, Xi , X 2 , • • •) has a well defined distribution. We 
shall be concerned with various modes of convergence of Xn to x. 

A . When P{ \ x^ — x \ > e) —> 0 as x, for any e > 0 , we shall say that 
Xn converges to x in 'probability. 

B. When E \ Xn — x —> 0 , as /i —► <», where 7 > 0 is fixed, we shall say that 
Xn converges to x in the mean of order 7 . Unless otherwise stated we shall in 
the sequel always consider the case 7 = 2 , and in this case we shall use the 
notation 

l.i.m. x„ = X. 

n-*oo 

C. When P(lim x„ = x) — 1 , we shall say that Xn converges with probability 

n-*oo 

one, or converges almo.^t certainly to x. 

With respect to the last definition, we may remark that the set defined by 
the relation lim Xn = x is always a Borel set in the space of our random sequence, 
so that the probability of this relation is well defined. In fact, this probability 
is given by the expression 

lim lim lim P [| x, — x | < - for j' = n, n + 1 , • • , n + p) 

m“*ao n-*oo p—oo \ TYi / 

where the limit process applies to a probability attached to a Borel set in a finite 
number of dimensions. The case of almost certain convergence is precisely 
the case when this expression takes the value 1 . 

Convergence in the mean of any positive order, as well as almost certain 
convergence, both imply convergence in probability, which may be w^ritten 
symbolically P —> A and C —^ A. Between B and C, there is no simple relation 
of this kind. Further, A and B both imply almost certain convergence for any 
partial sequence Xm , , • • • such that the subscripts increase sufficiently 

rapidly with h. 

If. PROBLEMS CONNECTED WITH THE .\DDrnON OF 
INDEPENDENT VARIABLES 

7. During the early development of the theory of probability, the majority 
of problems considered were connected with gambling. The gain of a player 
in a certain game may be regarded as a random variable, and his total gain in a 

* As 1 have already stated in a paper published in 1938, there is an error in the state¬ 
ment of this theorem given in my Cambridge Tract [9] Handom Variables and Probability 
Distributions. For the truth of the theorem, it is essential that ^n(f) should be supposed 
to converge to tpit) for every real t. However, in the particular case when the limit ip(t) 
is analytic and regular in the vicinity of f =* 0, it can be proved that it is sufhcieni to assume 
convergence in some interval | t | < a. 
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sequence of repetitions of the game is the sum of a number of independent 
variables, each of which represents the gain in a single performance of the game. 
Accordingly a great amount of work was devoted to the study of the probability 
distributions of such sums. A little later, prt)blems of a similar type appeared 
in connection with the theory of errors of observation, when the total error was 
considered as the sum of a certain number of partial eirors due to mutually 
independent causes. At first onlj* particular cases were considered, but gradu¬ 
ally general types of problems began to arise, and in the classical work of Laplace 
several results are given concerning the general problem to study the distribution 
of a sum 

= rCi + • • • -f 

of independent variables, when the distributions of the Xj are given. This 
problem may be regarded as the very starting point of a largo number of those 
investigations by which the modern Theory of Probability wuh created. The 
efforts to prove certain statements of Laplace, and to extend his results further 
in various directions, have largely contributed to the introduction of rigorous 
foundations of the subject, and to the development of the analytical methods. 
At the same time, more general types of problems have developed from the 
original problem, and the number and importance of pra(‘tical applications 
have been steadily increasing. 

8. Composition of distributions. Let .I’l and .1*2 l)t' two independent variables, 
with the d.f.’s F 1 and F2 , and the c.f.’s <pi and ^ , and let the sum xi + »r2 have 
the d.f. F and the c.f. <p. Then 

Fix) = f Fi(x — y) dFtiy) = f F^ix - y) dFiiy). 

J—oo J—00 

We shall say that F is the composition of Fi and F2 , and w rite this as a symbolical 
multiplication: 

F — Fi * F2 == F2 * Fi . 

To this symbolical multiplication of the d.f:s corresponds a real multiplication 
of the c.f.*s: 

(p{z) = <Pl{z)<P2{z) • 

The operation of composition is both commutative and associative, so that 
any symbolical product F = Fi*F2'- *Fn is uniquely defined and independent 
of the order of the components. When at least one of the components is con¬ 
tinuous (absolutely continuous), the same holds for the composite, and in 
many cases it is true that the composite is at least as regular as the most regular 
of the components (L 4 vy, [ 58 ], [ 63 ], etc.). However, this general statement 
does not hold generally, as is shown by an interesting example due to Raikov, 
[ 77 ], where Fi and F2 are integral analytic functions, while the composite F = 
Fi*F2 is not regular at the origin. 

It seems to be an important unsolved problem to find convenient restrictions 
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ensuring the validity of the above statements of the “smoothing effect” of 
the operation of composition. 

When F = Fi ♦ F2, we may say that F is “divisible” by each component Fi 
and F2 , and it seems natural to try to develop a theory of symbolical factoriza¬ 
tion for d.f.’s. In this connection, it is important to note that symbolical divi¬ 
sion is not unique. In fact, Khintchine has shown by an example that it is 
possible to find the d.f .’s F, Fi, F2, and Fs such that 

F = Fi ♦ F2 = Fi * Fs, 

while F2 7*^ F3. Another fundamental problem belonging to this order of ideas 
is to decide whether a given d.f. F is decomposable or not. F is called decom¬ 
posable, if there is at least one representation of the form F = Fi ♦ F2, where 
each component F„ has more than one point of increase. So far, this problem 
has only been solved in very special cases, and the general problem still re¬ 
mains open for research. A particular case of some interest would be to know 
if there exists an absolutely continuous and indecomposable d.f., such that 
F(a) = 0 and F(6) = 1 for some finite a and h. 

As soon as wc restrict ourselves to certain special classes of distributions, 
it is possible to reach results of a more definite character concerning the factori¬ 
zation problems. Some results of this type will be considered below. 


9 . Closed families of distributions. The fact that certain families of dis¬ 
tributions arc closed with respect to the operation of composition has played 
an important part in many applications. If Fi and F2 belong to a family of 
this character, so does the symbolical product F = Fi * F2. We first give some 
simple examples of such families. 

The normal distribution. The d.f. F has the form F ~ 4 ) 

<r > 0, and 




4}(x) = 


V2i 


r 

^2Tr i-oc 


-(«*/ 2 ) 


dt. 


The c.f. corresponding to F is c”*** *'***, and it follows that for any real mi , 
m2 and any positive <7i, 0-2 we have 


where 


m = mi -f m2, 


2 , 2 

= (Tl (T2 * 


The Poisson distribution. Here the d.f. is F = F{x; X, m, a) where X > 0 , 

a 5*^ 0 , and F is a step-function with a jump equal to ~ in the point x = 

^ -|_ yay where 1/ = 0 , 1 , • • • . The corresponding c.f. is and it 

follows that for any fixed a we have 

F(x; Xi, mi, a) ♦ F(x; X2, m2, a) = F{x\ Xi + X2, mi -f- m2, a). 
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r* 

The Pearson Type JII distribution. F — F{x\ a, X) — 377: / 

I (A) Jo 

, and for any fixed a > 0 and any 

positive Xi and A2 we have 

F{x; a, Xi) * F{x; a, X2) = F{x; a, Xi + ^2^- 

Stable distributions. We shall say that a closed family is stable, when all 
its members are of the form F{ax + 6), where F is a d.f., while a > 0 and b are 
constants. Obviously the normal family is an example of a stable family. It 
has been shown by I^^vy and Khintchine [ 49 ], that a d.f. F(x) generates a stable 
family when and only when the logarithm of its e.f. is of the form 

(9.1) log <piz) = piz - y\z r |-7j 0 )^ 

where a, 7, 5 are real constants such that 

0 < a ^ 2, 7 > 0, U I ^ 1, 

while 


{x > 0 ). The corresponding e.f, 


• A 



- log 1 ZI 

TT 


for a ^ 1 
for a = 1. 


For a = 2 we obtain the normal family. 

A more general and very important closed family is the family I of infinitely 
divisible distributions. A d.f. F belongs to I if to every n = 1, 2 , ••• there 
exists a d.f. G such that F = 6'^”^ where denotes the symbolical ?ith power 
of G. Obviously the family I is a closed family which contains all the families 
mentioned above. I^^vy [ 00 ], [ 03 ], has shown that F is infinitely divisible when 
and only when the hjgarithm of its e.f. is of the form 


log ,P(Z) = »iz - v + (e- - 1 - 


dM(u) 


izu \ 

iT^V 


dN(u), 


where 0 and 7 > 0 are real constants, while M(u) and N(u) are non-decreasing 
functions such that 


il/(—00) = iV(-|-oo) = 0 , 

J u^dM(u) < 00 and J dN{u) 


for any finite a > 0. When M and N reduce to zero, we obtain the normal 
family. When 7 = 0 and one of the functions M and N reduces to zero, while 
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the other is a step-function with a single jump equal to X at the point rr = a, 
we obtain a Poisson family. Generally, it follows from (9.2) that any infinitely 
divisible distribution may be regarded as a product of a normal distribution 
and a finite, enumerable or continuous set of Poisson distributions. 

The representation of log ^(2) in the form (9.2) is unique. It follows that 
the problem of finding all possible factorizations of an infinitely divisible d.f. F 
can be completely solved, as long as we restrict ourselves to factors which are 
themselves infinitely divisible. In fact, in order that 

F = 

where all three d.f .*s belong to /, it is necessary and suflicient that the logarithms 
of the corresponding c.f.’s should be of the form ( 9 . 2 ), with 

^ = ft + ft , 7 = 7i + 7i > 

M = Ml + il/2, TV = ATi + TVj. 

In the two simple cases of the normal and the Poisson distributions, the 
decompositions obtained in this way remain the only possible, even if we remove 
the restriction that the factors should belong to /. Thus in any factorization 
of a normal distribution, all factors are normal (Cramer, [8]), while in any fac¬ 
torization of a Poisson distribution, all factors belong to the Poisson family 
(Raikov, [ 75 ]). For the type III distribution, and the non-normal stable dis¬ 
tributions, however, the corresponding property does not hold. 

In some cases, an infinitely divisible distribution may be represented as a 
product of indecomposable distributions, or as a product of an indecomposable 
distribution and another infinitely divisible distribution. The results so far 
obtained in this direction (L6vy, [ 63 ], [ 64 ], Khintchine, [ 46 ], [ 47 ]; Raikov, [ 76 ]) 
are all concerned with more or less particular cases, and the general factoriza¬ 
tion problem for infinitely divisible distributions still remains unsolved. A 
particular case of some interest would be the case when the functions M and N 
are both absolutely continuous. There does not seem to have been given any 
example of this type, where a factor not belonging to I may occur.* 

Finally we mention a general theorem due to Khintchine, [ 46 ], which asserts 
that an arbitrary d.f. F may be represented in one of the forms 

F = G, F = ^ or F = (? ♦ /y, 

where G is infinitely divisible, while H is a finite or infinite product of inde¬ 
composable factors. This seems to be practically the only result so far known 
concerning the factorization of a general distribution. 

A certain number of the results mentioned above have been generalized to 
multi-dimensional distributions. 


■While the present paper was being printed, I have proved that such factors do occur, 
as soon as at least one of the derivatives M' and N* is bounded ^way from zero in some 
interval (—a, 0) or (0, a). 
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10. The Laws of large numbers. In modern terminology, the classical 
Bernoulli theorem may be expressed in the following way. Let Xi, X2, • * • be 
a sequence of independent variables, such that each x^ may only assume the 
values 1 and 0 , the corresponding probabilities being p and g = 1 — p. Then 
the arithmetic mean 


( 10 . 1 ) 


2 |i ^ Xi+ + Xn 

n n 


converges in probability to p, as n 00. 

Both classical and modern authors have laid down much work on the gen¬ 
eralization of this simple result in various directions. Generally, we shall say 
that a sequence of random variables XiyX^, • • • satisfies the Weak Law of Large 
Numbers if there exist two sequences of constants Oi, 02, • • • and 61, 62, • • • , 
such that Un > 0, and 

Zn — bn _ Xi +•• • + Xn — bn 

an an 


converges in probability to zero.. 

Let Xi y X2 • ‘ * ho independent variables, such that x, has the d.f. Fp(x). 
It has been shown by Feller [ 27 ] that for any given sequence ai, 02, • • • , the 
conditions 


( 10 . 2 ) 


E / dFp{x) = 0 ( 1 ), 

r-l ''1*1 >o» 

E [ x^dFp(x) = o(a\)y 
J\x\<an 


are sufficient for the validity of the weak law of large numbers, and that the 
corresponding sequence hi, ^>2, * • • can be defined by 


bn I xdFp{x), 

1—1 J\x\<an 

When there is a constant c > 0 such that for all v 


( 10 . 3 ) Fp(+ 0 ) > c, F,(- 0 ) < 1 - c, 


the conditions are also necessary. This theorem contains as particular cases 
all previously known results in this direction. A simple NS condition for the 
existence of at least one sequence 01,02, • • * such that 10.2 holds does not seem 
to be known. 

When the weak law is*satisfied, this means that, for any given € > 0 and for 
any fixed large w, there is a probability very near to 1 that the sum Zn = Xi + 
• * • + will fall l)etween the limits bn =t €o„ . The more stringent condition 
that, with a probability tending to 1 as n —> qc , will fall between the limits 

z — b 

bp db €ap for all values of p n is equivalent to the condition that - - - con- 

On 
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verges almost certainly to zero. When this holds, we shall say that the variables 
Xr satisfy the Strong Law of Large Numbers. The most important result so far 
known in this connection is concerned with the case an — n, and is expressed 
by the following theorem (Kolmogoroff, [ 52 ], [ 55 ]): 

When the Xp are independent and ( 10 . 3 ) holds, a sufficient condition for the valid¬ 
ity of the strong law with an = n consists in the simultaneous convergence of the 
two series 


S f dFn(x) and 2-i f 7?dFn(x). 

•^lxl>n n2J|x|<n 

Some improved conditions of tliis type have been given by Marcinkievvdez 
and Zygmund, [ 73 ], but the problem of finding NS condition for the strong 
law is still unsolved, even in the case On — n. 

Important generalizations of the laws of large numbers to cases when the 
Xp are not assumed to be independent have been given i.a. by Khintchine [ 44 ], 
L 4 vy [G 2 ], [G 3 ] and Lo^ve [G 7 ]. 

11 . The central limit theorem and allied theorems. It was already known 
to De Moivre that, in the case 10.1 of the Bernoulli distribution, the d.f. of 
the normalized sum 


Xi + • • • + Xn np 
\/ npq 

tends, as n —> qo, to the normal d.f. <f>ix). Considerably more general results 
in this direction were stated by Laplace. After a long series of more or less 
successful attempts, a rigorous proof of the main statements of Laplace was 
given in 1901 by Liapouneff, [ 65 ]. More general cases were later considered i.a. 
by Lindeberg [66], Ldvy [ 61 ], [ 63 ], IChintchinc [ 43 ] and Feller, [ 25 ]. The follow¬ 
ing final form of the Central Limit Theorem is due to Feller. 

Consider the e.xprcssion 

y - - - \ Zn l^n Xi “b * • * “h Xn 5n 

<“■» “• - - 5 ;^-- 

where the Xp are independent variables. We shall say that the Xp obey the 
central limit law, if the sequences {a„} and {bp] can be found such that the 
d.f. of Un tends to <l>(x) as n —> oo . In order to avoid unnecessary complica¬ 
tions, we shall restrict ourselves to sequences {a^} such that 


tty —> + 00 , 


OMl^l 

a. 


and we shall assume that the conditions ( 10 . 3 ) are satisfied. Then Feller’s 
theorem runs as follows: 
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The independent variables Xi, X 2 , • • • obey the central limit law ify and only if, 
there exists a sequence Qn ^ such that simultaneously 


( 11 . 2 ) 


i: f dF,(x) 0, 




|a|<«n 


dFp(x) 


—> 00. 


When these conditions are satisfied, explicit expressions for the an and 6„ can he 
obtained. 

Feller’s theorem gives a complete solution of the problem. However, we 
might still try to express in a more direct way the condition that the qn should 
exist. We may also ask what happens when the conditions (11.2) are not 
satisfied. Some particular cases of the latter question will be considered below. 
However, very few general results are known in this direction. 

The central limit theorem has been extended in various directions. Bern¬ 
stein [3], L^vy [62], [63], Lo5ve [67] and others have considered cases where the 
Xy arc not assumed to be independent. Important results have been reached 
but still much remains to be done. 

On the other hand, several authoi-s have considered symmetrical functions, 
other than sums, of n independent random variables. The problem of investi¬ 
gating the asymptotic behavio\ir of the distributions of such functions, as r? 
tends to infinity, is of great importance in the theory of statistical sampling 
distributions. It is known (c.f. e.g. Cramdr, [15]) that under certain general 
regularity conditions there exists a normal limiting distribution. However, it 
is also known that it is possible to give examples of particular functions (such 
as e.g. the function which is equal to the largest of the n vaiiables), Avhere there 
exist limiting distributions which are non-normal. The conditions under 
which this phenomenon may occur seem to deserve further study. 

A further problem belonging to the same order of ideas is to find a closer 
asymptotic representation of the d.f. of the standardized sum Zn than that pro¬ 
vided by the normal function 0(j*). Consider e.g. the simple case when the Xp 
are independent variables all having the same d.f. F{x) with a finite mean m, a 
finite variance <r^, and finite moments up to a certain order A* ^ 3. Let Gn{x) 
be the d.f. of the variable 


0:1 + • • • + Xn — nm 
a\/ n 

It then follows from a theorem of Cram6r [5], [9] that, as soon as the d.f. F{x) 
contains an absolutely continuous component, there is an asymptotic expansion 

(11.3) GM = + E 

Vmml n 

where the constant implied by the 0 is independent of n and x. Cramer has 
also given similar expansions in more general cases, and his results have been 



PROBLEMS IN PROBABILITY THEORY 


177 


further extended by P. L. Hsu [39], who deduces analogous expansions also for 
other functions than sums. The most general conditions under which expansions 
of this type exist are still unknown. 

It follows from (11.3) that the difference Gj,{x) — <l>{x) is, for any fixed x, 
of the order vT^ as r —► «. It is often important to know^ the as3rmptotic 
behaviour of Gn{x) when n and x increase simultaneously, and in that case (11.3) 
yields only a trivial result. This case has been investigated by Cram6r [10], 
and Feller [29], and the results so far obtained permit important applications to 
the so called law of the iterated logarithm (cf. below). However, it seems likely 
that similar results may be obtained in considerably more general cases than those 
hitherto investigated. 

A further interesting type of problems belonging to this order of ideas may 
be approached in the following way. Consider the variables (11.1) in the par¬ 
ticular case when i*i , j *2 , * * * are independent variables all having the same 
d.f. F(x). When the an and bn can be found such that the d.f. of the normalized 
sum Un tends to 0(a:), we shall say that F belongs to the domain of attraction of 
the normal law. Feller’s theorem gives a NF> condition that this should be so. 
Now when this condition is not satisfied, it may still occur that the an and bn 
can be so chosen that the d.f. of tends to a limiting d.f. '^(x), which is neces¬ 
sarily different from 4>(x). Then it is easily seen that ^(a*) must be a stable 
distribution, with its c.f. defined by (9.1), and it is natural to say that F belongs 
to the domain of attraction of NS and sufficient conditions that this should 
hold have been given by Doeblin [16], and Gnedenko [34]. When the On and 
bn cannot be found such that the d.f. of the normal sum Un converges to a limit, 
it may still be possible to obtain a limiting d.f. by considering only a partial 
sequence Un,, Wnj, * * • • Khintchine [47] has proved the interesting theorem 
that the totality of limiting d.f.’s that may be obtained in this way coincides 
with the class of infinitely divisible d.f.’s defined by (9.2). There are also 
further results in the same direction given by Bawly [2], Khintchine [44], L^vy, 
[61]-[63], and Gnedenko, [35]. 

12. The law of the iterated logarithm. Consider a sequence of independent 
variables Xi • • • , such that the mean Exn = 0 for all n, while the variances 
Ex\ == <t\ are finite. Put si == <r5 -h • + crl, and suppose that the variables 
obey the central limit law with On = «n, 5n = 0. (In particular this will be 
the case when all a:„ have the same distribution.) For any function ^(n) tending 
to infinity with n we then have 

(12.1) lim P(| Zn I > Sn^{n)) = 0. 

n-»oo 

On the other hand, if ^(n) tends to a finite limit > 0, the same probability 
has a positive limit. 

It seems natural to consider the relation within the brackets in (12.1) not 
only for a single large value of n, but to require the probability that this relation 
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holds simultaneously for an infinite number of values of n. The development 
of this problem has led to the so called law of the iterated logarithm. 

We shall in this respect use the follo\ving terminology due to L4vy. A non¬ 
decreasing positive function ^(n) will be said to belong to the lower class with 
respect to the variables Xn if, with a probability equal to one, there are infinitely 
many n such that 

I 2n I > Sn^(n). 

On the other hand, ^(n) will be said to belong to the upper class if the prob¬ 
ability of the same property is equal to zero. 

Every ^(n) belongs to one of these tw^o classes. This is a special case of the 
so called null-or-one law: if S is a Borel set in the spac‘e of the independent random 
variables , 0*2, • • • , such that any tw^o ])oints differing at most in a finite num¬ 
ber of coordinates either both belong to S or both belong to the complementary 
set, then P(<S) can only assume the values 0 or 1. 

It was proved by Kolmogoroff [51] that, subject to certain restiictions, the 
function 

^(?i) = Vc log log Sn 


belongs to the lower class for any c < 2, and to the upper class for any c > 2, 
w^hich may be expressed by the relation 


( 12 . 1 ) 


(“ 


P {lim sup 




1. 


s„ \/2 log log s„ 

More general results were proved by Feller [30], who proved i.a. that, subject to 
certain restrictions, ^{n) belongs to the lower or upper class according as 


( 12 . 2 ) 


2 ^(n)e- 

Sn 


•(^*(n)/2) 


is divergent or convergent (in certain special cases, this had been previously 
found by Kolmogoroff and Erdos [24]. Feller also pro\ed a more compli¬ 
cated result, w^hich contains the above as a particular case, and from which 
it follows that the simple criterion (12.2) no longer holds when the restrictions 
imposed in its proof are removed. 


13. Convergence of series. For any sequence of random variables Xn , the 
probability 

P Xn converges^ 

has a uniquely determined value. When the Xn are independent, it follows from 
the null-or-one law' that this probability is either 0 or 1. By a theorem of 
Khintchine and Kolmogoroff [48], the value 1 is assumed w'hen and only w'hen 
the three series 




dFn, 
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are convergent, where 

Xn when \xn\ ^ 1. 

Vn == 

0 when | Xn | > 1. 

For the case when the Xn are not assumed to be independent, various results 
have been given by L4vy [63] and others, but our knowledge of the properties 
of these series is still not very advanced. 

14. Generalizations. In several instances it has been pointed out above 
that the results concerning sums of independent variables may, to a certain 
extent, be extended to cases when the variables arc not independent. Generally 
the independence condition has then to be replaced by seme condition restricting 
the degree of dependence. Results of this type were first give by Bernstein 
[3], and then in more general cases by L6vy [62], [63], and Lo^ve [67]. However, 
this field has so far only been very incompletely explored. 

Similar remarks apply to the generalization of the various theorems quoted 
above to cases of variables and distributions in more than one dimension. 

III. STOCHASTIC PROCESSES 

16. The theory of random variables in a finite number of dimensions is able 
to deal adequately with practically all problems considered in classical prob¬ 
ability theory. Ho>vever, during the early years of the present century, there 
appeared in the applications various problems, where it proved necessary to 
consider probability relations bearing on infinite sequences of numbers, or even 
on functions of a continuous variable. 

The mathematical set-up required for the study of such problems involves 
the introduction of probability distributions in spaces of random sequences or 
random functions (cf. 5 above). Generally, any process in nature which can be 
analyzed in terms of probability distributions in spaces of these types will be 
called a stochastic process. It is convenient to apply this name also to the prob¬ 
ability distribution used for the study of the process. We shall thus say, e.g., 
that a certain random function x(t) is attached to the stochastic process which 
is defined by the probability distribution of x{t). In the majority of applica¬ 
tions, the variable t will represent the time, and w e shall often use a terminology 
directly referring to this case. However, there are also other types of problems 
in the applications {t may e.g. be a spatial variable in an arbitrary number of 
dimensions), and it is obvious that the purely mathematical problems connected 
with these classes of probability distributions will have to be considered quite 
independently of any concrete interpretation of the variable t or the funcion x{f). 

A well-known example of this type of problems is afforded by the Brownian 
movement. Let x{t) be the abscissa at the time ^ of a small particle immersed 
in a liquid, and subject to molecular impacts. In every instant, the quantity 
x(0 receives a random impulse, and the problem arises to study the behaviour 
of x{t). According as we are content to consider x{() for a discrete sequence 
of /-points, say for / = 0,1,2, • • • , or we wish to consider all positive values of t. 
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we shall then have to introduce a probability distribution in the space of the 
random sequence a;(0), a;(l), • • • , or in the space of the random function x(t), 
where ^ > 0. We may then discuss such questions as the distribution of x(t) 
for a given value of the joint and conditional distributions of x{t) for two or 
more values of t, and, in the case of a continuous variable tj continuity, differen¬ 
tiability and other similar properties of the random function x(t). 

Wiener [82], [83] (cf. also Paley and Wiener [74]) was the first to give a rigorous 
treatment of this process. He proved in 1923 that it is possible to define a 
probability distribution in a suitably restricted functional space, such that the 
increment Ax{t) = x{t + AO — x(t) is independent of x{t) for any At > 0. 
With a probability equal to 1, the function x(i) is continuous for all t > 0, and 
for any fixed t > 0, the random variable x{t) is normally distributed. 

Another example of stochastic processes studied at this stage occurs in the 
theory of risk of an insurance company. Let x{i) denote the total amount 
of claims up to the time t in a certain insurance company. As in the case of 
the Brownian movement, it may seem natural to assume that the increment 
Ax{t) is independent of x{t). On the other hand, x{t) is in this case an essen¬ 
tially discontinuous function, which is never decreasing, and increases only by 
jumps of varying magnitudes occuning for certain discrete values of /, which are 
not a priori known. Processes of this type were studied by F. Lundbei*g [09], 
[70], H. Cramer [6] and others. 

Further examples of particular processes were discussed in connection with 
various applications, but no general theory of the subject existed until 1931, 
when Kolmogoroff published a basic paper [63] dealing with the class of stochastic 
processes which will here be denoted as Markoff processes (Kolmogoroff uses the 
term ^‘stochastically definite processes”), of which the two examples mentioned 
above form particular cases. The theory of this class of processes was further 
developed by Feller [26], [28]. In 1934, Khintchine [42] introduced another 
important class of processes known as stationary processes. From 1937, the 
general theory of the subject was subjected to a penetrating analysis in a series 
of important works by Doob [18]-[22].’* 

16. Probability distributions in functional spaces. We have seen in 5 
above how a probability distribution in the space of all functions x{t) may be 
defined, when t varies in an arbitrary space T. Generally, we vshall here con¬ 
tent ourselves to consider the cases when T is the set of all real numbers, or the 
set of all non-negative real numbers. Most results obtained for these cases 
will be readily generalized to cases when i varies in a Euclidean space of a finite 
number of dimensions. 'On the other hand, when T is enumerable, say consist¬ 
ing of the points t = 0, ±1, ±2, • • • , so that we are concerned with a random 
sequence o^CO), x(±l), • • • , the results for the continuous case will generally 
hold and assume a simpler foi-m which will not be particularly stated herf/. 

• A further interesting paper by Doob has appeared while the present paper was being 
printed: '‘Probability in function space”, Bull. Amer. Math. Soc., Vol. 53 (1947), pp. 1&-30. 
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The case when T is a space of an infinite number of dimensions does not seem 
to have been considered so far. 

In the present paragraph, it will be convenient to assume the function x{t) 
to be real-valued, but the generalization to a complex-valued x{t) requires 
only obvious modifications. In the sequel we shall sometimes consider the 
real-valued and sometimes the complex-valued case, according as the occasion 
requires. 

Let now X be the space of all real-valued functions x{t) of the real variable 

where — oo < t < oo . According to 5, a probability measure P{S) is uniquely 
defined for all Borel sets aS in X by means of the family of joint distributions 
of all finite sequences • • • , xQn). In fact, P{S) can be defined for a more 
general class of sets than the Borel sets. For any set S in X, wo may define 
an outer P-measure P(S) as the lower bound of P(Z) for all sums Z of finite or 
enumerable sequences of interv’’als, such that S Cl Z. Further, the inner P- 
measure P(<S»)is defined by the relation P(S) = 1 — P(X — S). When the 
outer and inner measures are equal, S is called P-measurable, and P(S) is defined 
as their common value. Any P-measurable set differs from a Borelset by a 
set of P-measure zero. 

In many cases, this definition will be sufficient for an adequate treatment 
of the problems that we wish to consider. However, in other cases we encounter 
certain characteristic difficulties, which make it desirable to consider the pos¬ 
sibility of amending the basic definition. Thus it often occurs that we are 
interested in the probability that the random function x(t) satisfies certain 
regularity conditions in a non-enumerable set of points (. We may, e.g., wish 
to consider the probability that x{t) is continuous for all that x(t) should 
be Lebesque-measurable for all t, that x{t) ^ k for all t, etc. Let S denote the 
set of all functions satisfying a condition of this type. It can then be shown 
that the inner measure ^(S) is always equal to zero so that S is never measur¬ 
able, except in the (usually trivial) ca.se when P(aS») = 0. 

Consequently many interesting probabilities are left undetermined by the 
general definition of a probability distribution in X given above. The pos¬ 
sibility of modifying the definition so as to enable us to study probabilities of 
this type has been thoroughly investigated by Doob [18]. He considers a 
subspace Xo of the general functional space X, where Xo is chosen so as to 
contain only, or almost only, “desirable” functions, i.e. functions satisfying 
such regularity conditions as seem natural with respect to the problem under 
investigation. We start from a given probability measure P{S) in X, and ask 
if it is possible to define a probability measure in the restricted space Xo, which 
corresponds in some natural way to the given distribution in X. Let So be 
a set in Xo, and suppose that it is possible to find a P-measurable set aS in X 
such that aSXc = So . According to Doob, a probability measure Po in Xo 
is then uniquely defined by the relation 

PoiSo) = P{S) 
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if and only if the condition 

P(Xo) = 1 

is satisfied. 

The problem is thus reduced to finding a subspace Xo of outer P-measure 1, 
such that Xo contains only functions of sufficiently regular behaviour. When 
this can be done, we can restrict ourselves to consider only functions x{t) be¬ 
longing to Xo , the probability distribution in this space being defined by the 
measure Po. We shall then say that x{t) is a random function, attached to a 
stochastic process with the restricted space Xo . Doob has obtained a great 
number of interesting results in this connection, e.g. with respect to the problem 
of choosing Xo such that it contains almost only I^besqiie-measurable functions, 
or such that the probability of the relation x{t) ^ k has a well-defined value for 
all k. In particular he has shown that the last problem can be solved for 
any given P-measure. However, our knowledge cf the various possibilities 
which exist with respect to the choice of Xo is still very incomplete, and it seems 
likely that further important results may be reached along this line of research. 

An alternative method of introducing probability distributions in functional 
spaces has been used by Wiener [82], [83], (cf. also Paley and Wiener, [74]). 
Consider a given probability measure IT in an arbitrary space 11, defined for all 
sets 2 of an additive class C. Let x(/, w) denote a function (real- or complex¬ 
valued, as the case may be) of the arguments i (real) and o) (point in H), such that 
x{t, w) for every fixed t becomes a C-measurable function of w. On the other 
hand, when w is fixed, x{t^ w) = x{t) reduces to a function of the real variable t. 
Let Xo denote the set of all functions x{t) corresponding in this way to points of 
11. Further, let Sq = ^SXo , where is a Borel set in X, and let 2 denote the set 
of all points w such that x{t, w) C 60. Then 2 belongs to (\ and a probability 
measure Po in the functional space Xo is uniquely defined by the relation 

( 16 . 1 ) Po(^o) = n(S). 

The relations between the two modes of definition ha\'(» been discussed by 
Doob and Ambrose [23] who have shown that they are lai-gely equivalent. 
However, it seems likely that in particular problems the one or the other pro¬ 
cedure may sometimes be the more advantageous, and further investigations 
on this subject seem desirable. 

17. Processes with a finite mean square. Consider a stochastic process 
defined by a probability measure P(S) in the space X of all complex-valued 
functions x{f) of the real variable L For any fixed U , the random variable 
x{to) is then a complex-valued function of the variable point x{t) in the space 
X, i.e. a point Qt^ in the space Q of all complex-valued functions defined on X. 
When Ui varies, the point Qt^ describes a “curve” in 12, which then corresponds 
to our stochastic process. 
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Suppose, in particular, that the mean square 

£;|*(or= f ix«)pdP 

Jx 

is finite for any fixed value of t. This implies that for fixed t the function 
x(t) belongs to over X, relative to the probability measure P. The random 
variable x{t) may then be regarded as an element of the Hilbert space H of all 
complex-valued functions / belonging to L 2 over X, the inner product (/, g) of 
two elements / and g being defined by the relation 

if,g) = jjddP = Eifg). 

The stochastic process to which x(t) is attached then corresponds to a “curve*’ 
in 11 (Kolmogoroff, [56], [57]), so that the well-known theory of Hilbert space is 
availaiile for the study of the process. In particular, convergence in the usual 
metric of Hilbert space is equivalent to convergence in the mean of order 2 for 
random variables. 

Let Hx be the smallest closed linear subspace of II which contains all elements 
of the form aix(h) + . • • + anx{tr). If the covariance function 

r{l,u) = (^(0, x{u)) = E{x{t)^)) 

is continuous for all real values of t and then x{t) x{U) in the mean, as 
t to y and we shall say that the process x{t) is continuous. For any continuous 
process, //* is separable. When g{t) is a continuous non-random function of t, 
and x{t) is attached to a continuous stochastic process, the Riemann-Darboux 
sums formally associated with the integral 

[ g(t)x(t)dt 

da 

are easily shown to tend to a limit 1/, which is an element of , i.e. a random 
variable. By definition, we may identify the integral with this variable Py 
and this integral will possess the essential properties of the ordinary Riemann 
integral (Cram6r, [12]). 

The application of the theory of Hilbert space to stochastic processes seems 
to open very interesting possibilities. Some applications to particular classes 
of stochastic pi-occsses will be mentioned below. Futher important results be¬ 
longing to this order of ideas will be given in a work by K. Karhunen [40], which 
is in course of publication. 

18. Relations to ergodic theory. There is a close connection between the 
theory of stochastic processes and ergodic theory. In ergodic theory, as sum¬ 
marized e.g. in the treatise of E. Hopf [38], we consider an arbitrary space 12, 
and a probability measure 11, defined for all sets S belonging to the additive 
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class C. We further consider a one-parameter group of one-one transformations 
of into itself (a in Q) such that the transformation corresponding to 

the parameter value t takes the point to = wo into while (co<)« = cof+„. Let 
/(w) be a given function, defined throughout 12, and such that /(co<) is C-measur- 
able for every fixed t. The well-known ergodic theorems due to von Neumann, 
Birkhoff, Khintchine and others are then concerned with the asymptotic 
behaviours of mean values, which in the classical cases are of the types 

SM +fM +-♦ + /(C0.-.1) 
n 
or 

ll’/Mdi. 

as n or T tends to infinity. (In the case of the latter expression, it is necessary 
to introduce some additional condition implying measurability in t.) 

Writing x(t, w) = /(o^i), it is seen that to a given transformation group w —+ cot 
and a given function/(w), there corresponds a stochastic process in the sense of 
W'^iener’s definition (cf. 16). The space Xo of this pn)cess consists ( f all functions 
x(i) representable in the form x(t) == f(cot), when w = a>o varies over 12. The 
corresponding probability measure Po is defined by (16.1). 

Thus any of the above-mentioned ergodic theorems may be expressed as a 
theorem concerning “temporaT’ mean values of the types 

3’(0 ) + x(l) -f * ♦ » + 3r(n — 1) 
n 
or 

fi 

If, according to some reasonable convergence definition, we may assign a limit 
to either of these expressions, as n or T tends to infinity, this limit will be a 
random variable, and it is important to find conditions which imply that this 
variable has a constant value for “almost alL’ functions x(0, i.e. for all x(0 
except at most a set of Po-measure zero. 

In the particular case when a;(0), a;(l), • • • are independent variables all 
having the same distribution, the classical ergodic theorems yield simple cases 
of the laws of large numbers (cf. 10). The mean ergodic theorem of von Neu¬ 
mann gives the weak law', while the Birkhoff-Khintchine theorem gives the 
strong law. Some more general results belonging to this order of ideas will be 
mentioned in the sequel. 

It will be seen that the two theories are largely equivalent, and it seems 
likely that further comparative studies of the methods will be of great value to 
both sides. 

19. Markoff processes. Consider now a stochastic process, defined by a 
probability measure P(S) in the space X of all real-valued functions x(f) of the 
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real variable i. For any ii < h i there is a certain conditional probability 
P(x{t 2 ) C S I x{ti) = Oi) of the relation x{t 2 ) C relative to the hypothesis 
that x{ti) assumes the given value ai. Suppose now that this conditional prob¬ 
ability is independent of any additional hypothesis concerning the behaviour of 
x(t) for ^ , so that we have e.g. for any k < k < k and for any a© 

P{x{k) C iS 1 x{k) = fli) = P{x{Q C S I x{k) = di , = oo). 

In this case the process is called a Markoff process. 

The general theory of this type of processes, which forms a natural gen¬ 
eralization of the classical concept of Markoff chains, has been studied in basic 
works by Kolmogorrff [53] and Feller [26], [28]. Writing 

P(x{f) ^ i I = Oo) = F({; k Oo, <o), 

where to < k F will be the distribution function of the random variable x{t)i 
relative to the hypothesis x(fo) - cto. Then F satisfies the Chapman-Kol- 
mogoroff equation 

(19.1) F({;ao, io) = f F(^; k) di,F(r}; k f Qa , k), 

whi<;h expresses that, starting from the state x(/o) = Oo, the state x{t) g f 
must be reached by passing through some intermediate state x{k) = where 
to < k < t. Subject to certain general conditions, it is possible to show that 
any solution of this equation satisfies certain integro-differential equations, 
which in some important cases reduce to partial differential equations of para¬ 
bolic type, and that the d.f. F is uniquely determined by these equations. How¬ 
ever, the general conditions mentioned above are in many cases difficult to apply 
to particular classes of processes, and it would be important to have further 
investigations concerning these questions. 

Markoff processes (not belonging to the subclass of differential processes, 
which will be considered in the following paragraph) appear in several important 
applications, e.g. in the theory of cosmic radiation, in certain genetical problems, 
in the theory of insurance risk etc. In these cases, we are often concerned with 
the class of purely discontinmus Markoff processes, where the function x{t) 
only changes its value by jumps. If, in addition, there are only a finite or 
enumerable set of possible values for x{t), the Chapman-Kolmogorcff equation 

(19.1) reduces to 

(19.2) iraik , ^) = 2 k)‘irjk{k , t), 

i 

where ir,ji(<o, t) denotes the ‘Transition probability*^ i.e. the probability that 
x{t) will be in the kth state at the time k when it is known to have been in the 
ith state at the time k . In matrix form, this equation may be written 

(19.3) n(^o, 0 = Uik^kMk. t), 
where n denotes the matrix of the ir** . 
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When only a sequence of discrete values of t are considered, we have here 
the classical case of Markoff chains, which has received a detailed treatment 
in the well-known book by Fr^chet [32] (cf. also Doob, [19]). The case when t 
is a continuous variable has been treated by Feller [28], 0. Limdberg [71], 
Arley [1], and other authors. Some of the most important problems of this 
branch of the subject are concerned with the existence of a unique system of 
solutions of (19.2) or (19.3), and with the asymptotic behaviour of the solu¬ 
tions for large values of ^ — /o. Though important results have been reached, 
there still remains much to be done here, and the same thing holds a fortiori 
with respect to the analogous problems for general Markoff processes. 

20. Differential processes. A particularly interesting case of a Markoff 
process arises when, for any A/ > 0, the increment Ax(/) — Qc{t -f AO — x(J) 
is independent of x{t) for r ^ The process is then called a differential process. 
Some of the earliest studied stochaslic processes belong to this class, which 
contains in particular the tno examples discussed above in 15. Further cases 
of such processes arise e.g. in the theory of radioactive disintegration and in 
telephone technique. 

Tjet us suppose thai a'(0) is identically equal to zei-o, and that the process is 
uniformly continuous in probability in every finite interval 0 g t ^ T, i.e, 
that for any fixed positive e 

P{ I x{t + At) - x{t) 1 > €) -^ 0 

as A/ 0, uniformly for 0 ^ ^ 7' Then it follows from the works of L6vy, 

[60], [63], Khintchine [47] and Kolmogoroff [54] that, for any ( > 0, the random 
variable x{t) has an infinitely divisible distiibution, A\ith a characteristic func¬ 
tion fp(z; f) given by (9.2), where /9,7, M(ti) and N{u) may depend on L 

In the particularly important case when the distiibution of the increment 
x(t -f- AO = x{t) does not involve I, but only depends on the length At of the 
interval, we say that the i)rocess is tempo?ally homogeneous, and in this case 
we have 


log ^(2; 0 = tliy^ipiz; 1), 

so that we obtain the general formula for (p{z; l) simply by replacing in (9.2) 

7, M{u) and Niu) by t/3y ^7, txM{u) and lN{u) respectively. 

When t 00, or / —^ 0, the appropriately normalized distribution of x{t) 
tends, under certain conditions, to a stable distribution (Cramer [7], Gne¬ 
denko [36]). When this limiting distribution is normal, there are sometimes 
even asymptotic expansions analogous to (11.3). Still, the problem of the 
asymptotic behaviour of the distribution for large t does not seem to be definitely 
cleared up. 

Khintchine [41] and Gnedenko [37] have given interesting generalizations 
of the law of the iterated logarithm (cf. 12) to processes of the type considered 
here. 
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The continuous process discussed in 15 in connection with the Brownian 
movement corresponds to the temporally homogeneous case when M(tt) and 
N(u) all reduce to zero, so that 

which shows that the distribution of x(t) is normal, with mean zero and vari¬ 
ance 2'yL 

On the other hand, in the applications to the theory of insurance risk, y is 
zero, while M(u) and N(v) are connected with the distribution of the various 
magnitudes of claims. In this type of applications, it is often very important 
to find the probability that x{t) satisfies an inequality of the form 

x(t) < a + hi 

for all values of (. It follows from the discussion in 1() that the definition of 
a probability of this type is somewhat delicate. The problem, which can be 
regarded as an extended form of the classical problem of “the gambler’s ruin,” 
has been solved in certain particular cases. It leads to integral equations, 
which in the simplest case are of the Volterra, in other cases of the Wiener- 
Hopf type (Cramer [G], [13], Segerdahl [79], Tiicklind [81]). 

21. Orthogonal processes. Consider now the case of a complex-valued 
x(t)f and suppose that E | .t(/) is finite for all (. Without restricting the gen¬ 
erality, we may assume that Ex(0 = 0 for all t. 

Suppose now that instead of requiring, as in the case of a differential process, 
that the variables x(r) and Ax{i) should be independent when r ^ we only 
lay down the less stringent condition that these variables should be non-cor- 
related, i.e. that 

E(x{T)Ax{t)) = 0 . 

We then obtain a process which is no longer necessarily of the Markoff t3rpe. 
The condition implies that, for any two disjoint interv^als (<i, ^2) and (tz , U)f 
we have 

E[{x{k) - x{fi)){x(u) - x^)] = 0, 

so that the ^‘chords” corresponding to two disjoint ^‘arcs” of the curve in 
Hilbert space representing the process arc always orthogonal (Kolmcgoroff 
[56], [57]). A process of this type may accordingly be called an orthogonal 
process. 

For a process of this type we have, writing E | x{t) f = F((), F(t + At) — 
F(t) = E\x{t + At) — x{t) 1*, so that F{t) is a never decreasing function of 
If F(t) is bounded for all t, we shall say that the orthogonal process is bounded. 
For a bounded orthogonal process, the Stieltjes integral 

[ g{t) dx{i), 

j—» 
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where g{t) is bounded and continuous, may be defined as the limit in the mean 
of sums of the form 

p 

22. Stationary processes. When we are concerned with a process representing 
the temporal development of a system governed by laws which are invariant 
under a translation in time, it seems natural to assume that the joint distri¬ 
bution of any group of variables of the form 

( 22 . 1 ) xih + r), ••• , x{tn + r) 

is independent of r. A process satisfying this condition will be called a sta¬ 
tionary process. If a stochastic process is defined by means of a “flow” w 
in a space 12 (cf. 18), the process will be stationary when and only when the 
corresponding flow is measure-preserving, i.e. if the transformation co —> Wf 
changes any C-measurable set S into a set of the same measure. 

Under appropriate conditions with respect to the measurability of a;(0, the 
Birkhoff-Khintchine ergodic theorem holds for a stationary process, i.e. there 
exists a random variable y such that we have 

(22.2) Po ^lim ^ x(t) dt = ^ = 1, 

r-*oo 

where Po is the probability measure in a suitably restricted space in the sense 
of Doob. Further work seems to be required here, in order to make the situa¬ 
tion quite clear, also with regard to metric transitivity. 

For a stationary process, any finite moment of the joint distribution of the 
variables ( 22 . 1 ) is obviously independent of r. Suppose now that we only re¬ 
quire that this invariance under translations in time should hold for moments 
of the first and second order of the joint distributions, which are assumed to 
be finite. The wider class of processes obtained in this way may be called 
stationary of the second order. Processes of this type have been studied for the 
first time by Khintchine [42]. We shall assume that x{t) is complex-valued. 
Without restricting the generality, we may further assume that Ex{t) = 0 for 
all t. The product moment E{x{t)x{;u)) will then be a function of the difference 
t — u: 

(22.3) E{x(t)^ = R{t - m). 

Assuming, in addition, that R{t) is continuous at ^ = 0, it follows that R{t) 
is continuous for all t, and the process is continuous in the sense of 17. It was 
shown by Khintchine that a AT/S condition that a given function R{t) should 
be associated with a second order stationary and continuous process by means of 
the relation (22.3) is that we should have 

U{f) = r e'^^dFix) 

J^oo 


(22.4) 
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for all t, where the spectral function F{x) is real, never decreasing and bounded. 
In particular, we ha^'o 

/^(+ cc) - F{^ X) = /^(O) = E I x{t) = c\ 

Khintchine’s condition for /?(/) was generalized by Cramer to the case of an 
arbitrary number of processes a-i(/), • • • , Xn{t), such that the product moments 
E{xi{t)xj{ii)) are functions of the difference i — u. The corresponding spectral 
functions Fij{x) arc in general complex-valued and of bounded variation. Fur¬ 
ther, the expression (Cramer, [12]) 

n 

ZiZjAFi,, 

i.i-1 

where AF^j = Fij(h) — Fij(n) is, for any a < b, a, non-negative Hermite form in 
the variables Zi . This result is (*losely connected with a theorem on Hilbert 
space considered by Kolmogoi-off and Julia. It is further shown that, to any 
given functions (f, j — 1, • • • , n), satisfying these conditions, we can 

always find n processes Xi(t), • • • , Xn(t) such that the joint distribution of any set 
of variables Xi(tj) is always normal, while the covariance functions Rij{t — u) = 
E{Xt{t)xj{u)) are given by the expression 

RiiO) = 

For a process x{t) which is continuous and stationary of the second order* 
with Ex{t) = 0 for all /, we have the mean crgodic theorem 

(22.5) f e~’"'x(l)dt = y 

T-*oo Zi J—T 

for any real X. The random variable y has the mean 0 and the variance F(\ -f 0) 
— F{\ — 0), where F is the spectral function appearing in (22.4). If X is a 
point of continuity for F, it thus follows that y = 0 with a probability equal 
to 1. On the other hand, if X is a discontinuity, y has a positive variance. Let 
Xi, X 2 , • • • be all the discontinuities of F{x ), and let ai, ^ 2 , • • • be the cor¬ 
responding saltuses, while t/i, 2 / 2 , • * • arc the limits in the mean obtained from 

(22.5) for X = Xi , X 2 , Then two different ?/,• are always orthogonal; 

Eiyjyk) = 0 for j 9 ^ k, and we have 

(22.6) x{t) = E + m, 

P 

where Ff (/) = 0 and 

Ei{«)r = 

V 

If F{x) is a step-function, wc have == ^p , and it follows that {(<) = 0 
with a probability equal to 1, so that (22.6) gives a “stochastic Fourier expan¬ 
sion*' of x{t) (Slutsky, [80]). 
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Even wlien F{x) is arbitrary, we can obtain a spectral representation of a?(0 
generalizing (22.6). In fact, it can be shown (Cramer, [14]) that x{t) can always 
be represented by a Fourier-Stieltjos integral 

(22.7) x(0 = r dziu), 

J—CO 


where z{u) is a random function attached to a bounded orthogonal process 
(cf. 21), such that 

E I z{u -b Au) — z{u) f = F{u + Au) — F{u). 


Conversely, we have 


^00 —_ /)“***“ 

(22.8) z(u 4- Au) — z(u) = “ j --•'*‘(0 dt, 


so that there is a one-one correspondence between .r(/) and A 2 :(w). The integrals 
(22.7) and (22.8) are defined as limits in the mean, as shown above in 17 and 21. 
These results are in close correspondence with generalized harmonic analysis for 
an arbitrary function, as developed by Wiener [831 and Hochncr [4]. The spec¬ 
tral representation of a stochastic process ha.'*’ important applications, some of 
which will be considered in a forthcoming paper by Karhunen [40]. An exten¬ 
sion of the spectral representation to a moie gen(*ral class of piocesses has been 
given by Locve [08 j. 

When, in particular, the x{t) process is such that the joint distribution of any 
group of variables .r(/i), • • • , x{in) is normal, it follows that any increment 
Az(u) is normally distributed. Since two uiicorrolatcd normally distributed 
variables are always independent, it follow's that in this case the z{u) process 
is a differential process with normally distributed increments. Important 
results for this case have recently been given by Doob |22]. 

The properties of continuity, differentiability etc. for processes of the type 
here considered arc still incompletely known, and further work is required. 
A further group of important unsolved problem*^ are connected with an inter¬ 
esting decomposition theorem by Wold [84], which holds for processes with 
a discrete time variable. The generalization of this theorem to the continuous 
case does not seem to have so far been given in a final h)rm. 
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THE ESTIMATION OF DISPERSION FROM DIFFERENCES' 

By Anthony P. Morse^ and Frank E. Grubbs 
Ballistic Research Laboratory, Aberdeen Proving Ground, Maryland 

Summary. The estimation of variance by use of successive differences of 
higher order is discussed in this paper. Heretofore, attention has been focused, 
in published works, on estimates of variance obtained by employing the sum of 
squares of deviations from the mean and also by using mean square successive 
differences of the first order [1], [2], [3], [9]. A concise description of the method 
employing differences of any order with appropriate formulae for the precision 
of estimates so obtained and also a practical example on the use of the technique 
are given in section 11. Fundamental contributions to the estimation of 
variance from higher order differences, a study of the efficiency of the technique 
and proper orientation of the subject matter in the field of mathematical statis¬ 
tics are given in sections 2-10 of the paper. 

1. Introduction. It frequently happens that successive observations, made 
at regular intervals of time, are subject to the same standard error while the 
means of the populations from which they are drawn display some kind of trend. 
The type of trend we speak of is brought about because of the manner in which 
we have to take measurements or because of variations in the measuring tech¬ 
nique itself; or, again, the trend may be characteristic of the thing we are meas¬ 
uring. In any event, we may desire to eliminate the trend in order to study 
residual effects. As an example, it is desirable in the field of ballistics to evaluate 
the dispersion of machine guns firing from a moving airplane. 

It may also happen that it is either inexpedient or impossible to estimate the 
standard error of the observations by the method of least squares, for in a large 
number of cases the type of trend is unknown. In this event a method employing 
differences of an appropriate order may prove valuable. The method consists 
merely of arranging the data in a vertical column in the order in which the obser¬ 
vations were taken and then forming difference columns in the usual way of 
order 1,2, up to say 5 or some other number depending on the peculiarities of the 
problem at hand and the number of the original observations. Next, sum the 
squares of the numbers in each column and divide the sum of squares of the pth 

order differences by (n — p) • When n> 2 and p> I, the numbers thus 

arrived at are all unbiased estimates of the population variance for the case 
where all the observations have the same expected value. In section 11 at the 

^This paper is based substantially on a Ballistic Research Laboratory Report [10] 
of the same subject by Morse and has been prepared for publication by Grubbs at the sug¬ 
gestion of R. H. Kent. The authors are grateful to J. V. Lewis and H. L. Meyer for their 
many and varied comments, criticisms and suggestions. 

• Now at the University of California, Berkeley, California. 
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end of the paper will be found a summary of this method, formulas by which 
the precision of the estimate of the variance <r* may be determined, and an exam> 
pie displaying the stability of this estimate with respect to p. 

If a strong trend is present then the method of first differences will obviously 
yield an estimate of variance which is fictitiously large and the temptation to 
pass to higher order differences may quite reasonably be yielded to. As a matter 
of fact, unbiased estimates may be hoped for from pth order differences whenever 
there is good reason to suppose that the pth derivative of the trend function is 
small most of the time. However, even in the case of a sinusoidal trend where 
all derivatives have the same magnitude one may obtain good results frcm higher 
differences provided there are at least seven observations in each interval of 
length one period (see section 5 and Table II below). In connection with trends 
such as the sinusoidal type, the hopelessness of getting, say, even a fifth degree 
polynomial to fit over an interval of, say 20 periods is rather evident. It is 
for the above reasons that estimation of variance from higher order differences 
deserves consideration. 

2. Historical comment. A brief historical development of the interest in 
successive differences as a means for estimating dispersion is given in [3]. This 
paper discusses the statistic 



suggested by “Student** [W. S. Gossett] and E. S. Pearson and points out the 
relevant work of Jordan, Helmert, Vallier, Cranz, and Becker. It seems that 
Jordan devised methods based on sums of powers of the differences, whereas 
Helmert gave more careful consideration to the case of the first power, i.e. the 
sum of absolute differences. Reference [3] points out, however, that in these 
two cases all the n(n — 1) /2 differences that can be established frcm a sample of 
n observations were included in the estimates of dispersion recommended by 
Jordan and Helmert, so that the estimate was of no value in reducing the effect 
of a trend. Continuing the remarks of [3], we learn that in ballistics Vallier 
appears to have been the first to estimate dispersion from successive differences 
and that Cranz and Becker commended the mean successive difference 


Ed = S * - Xi I 

n — 1 

in estimating dispersion in range of guns since they were aware of variable ex¬ 
ternal effects (such as tail winds) on a projectile. In this country, Bennett [I] 
appears to have suggested the use of successive differences independently of 
European ballisticians. In this connection, Bennett suggested that the probable 
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TABLE 1 


The Efficiency, lT(n, p), of As An Esliniale of <r* 


/ 

1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

1.00000 










3 

.80000 

.50000 









4 

.75000 

.46154 

.33333 1 








5 

.72727 

.46552 

.32000 

.25000 







6 

.71429 

.47213 

.33149 

.24427 

.20000 






7 

.70588 

.47771 

.34453 

.25510 

.19672 

.16667 





8 

.70000 

.48214 

.35537 1 

.26871 

.20633 

.16471 

.14286 




9 

.69565 

.48568 

.36408 

.28071 

.21888 

.17274 

.14159 

.12500 



10 

.69231 1 

.48855 

.37113 

.29071 

.23058 

.18385 

.14830 

.12414 

.11111 


11 

.68966 

.49091 

.37691 ! 

.29904 

.24070 

.19476 

.15802 

.12978 

.11050 

.10000 

12 

.68750 

.49288 

.38173 

.30602 

.24934 

.20450 

.16798 

.13827 

.11529 

.09955 

13 

.68571 

.49455 

.38580 

.31194 

.25672 

.21300 1 

.17714 

.14729 

12271 

.10366 

14 

.68421 

.49598 

.38928 

.31701 

.26308 

.22039 

.18530 

.15581 

.13086 

.11018 

15 

.68203 

.49722 

.39228 

.32139 

.26859 

.22684 

.19250 

.16353 

.13874 

.11754 

16 

.68182 

.49831 

.39490 

.32522 

.27342 

.23251 

.19887 

.17045 

.14601 

.12481 

17 

.68085 

.49926 

.39721 

.32859 

,27767 

.23752 

.20452 

.17664 

.15260 

.13162 

18 

.68000 

.50011 

.39925 

.33158 

.28145 

.24197 

.20956 

18218 

.15855 

.13787 

19 

.67925 

.60087 

.40107 

.33424 

.28482 

.24595 

.21407 

.18715 

.16393 

.14356 

20 

.67857 

.50155 

.40271 

.33663 

.28784 

.24953 

.21813 

.19164 

.16879 

.14875 

21 

.67797 

.60216 

.40419 

.33880 

.29058 

.25276 

.22181 

.19571 

117321 

.15347 

22 

.67742 

.50272 

.40553 

.34075 

.29306 

.25569 

.22515 

.19941 

.17723 

.15778 

23 

.67692 

.50323 

.40675 

.34254 

.29532 

.25837 

.22819 

.20279 

.18091 

.16173 

24 

.67647 

.50370 

.40787 

.34417 

.29739 

.26082 

.23098 

.20588 

.18428 

.16535 

25 

.67606 

.50413 

.40889 

.34567 

.29929 

.26307 

.23354 

.20873 

.18738 

.16869 

26 

.67568 

.50452 

.40984 

.34706 

.30104 

.26514 

.23590 

.21135 

.19024 

.17177 

27 

.67533 

.50489 

.41071 

.34833 

.30266 

.26705 

.23809 

.21378 

.19289 

.17463 

28 

.67500 

.50523 

.41152 

.34951 

.30416 

.26884 

.24012 

.21603 

.19535 

.17728 

29 

.67470 

.50555 

.41228 

.35062 

.30555 

.27049 

.24200 

21812 

.19764 

.17975 

30 

.67442 

.50585 

. 11298 

.35165 

.30686 

.27203 

.24375 

.22007 

.19978 

.18205 

31 

.67416 

.60612 

.41363 

.35260 

.30807 

.27347 

.24539 

.22190 

.20177 

.18420 

32 

.67391 

.60638 

.41425 

.35350 

.30921 

.27482 

.24693 

.22361 

.20364 

.18622 

33 

.67368 

.60662 

.41482 

.35434 

.31027 

.27608 

.24837 

.22521 

.20539 

.18811 

34 

.67347 

.50685 

.41536 

35513 

.31128 

.27727 

.24973 

.22672 

.20704 

.18989 

35 

.67327 

.50707 

.41587 

.35588 

.31222 

.27839 

.25101 

.22814 

.20859 

.19157 

36 

.67308 

.50727 

.41635 

.35658 

.31312 

.27945 

.25221 

.22949 

.21006 

.19315 

37 

.67290 

.50746 

.41671 

.35724 

.31396 

.28045 

.25335 

.23075 

.21145 

.19465 

38 

.67273 

.60764 

.41724 

.35787 

.31476 

.28140 

.25443 

.23195 

.21276 

.19606 

39 

.67257 

.60781 

.41765 

.36847 

.31551 

.28229 

.25545 

.23309 

.21401 

.19741 

40 

.67241 

.60797 

.41804 

.35904 

.31623 

.28314 

.25642 

.23417 

.21519 

.19868 
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TABLE I —Continued 


n \ 

\ 

1 

2 

3 

4 

5 

i 

6 

7 

8 

9 

10 

42 

.67213 

.50828 

.41875 

.36009 

.31756 

.28472 

.25822 

.23617 

.21738 

.20105 

44 

.67188 

.50855 

.41941 

.36104 

.31877 

.28615 

.25986 

23799 

.21937 

.20320 

46 

.67164 

.50880 

.42000 

.36191 

.31987 

.28745 

.26135 

.23965 

.22118 

.20516 

48 

.67143 

.60903 

.42055 

.36271 

.32088 

.28866 

.26271 

.24117 

.22284 

.20695 

60 

,.67123 

.50925 

.42106 

.36343 

.32180 

.28975 

.26397 

.24256 

.22437 

.20860 

62 

.67105 

.50944 

.42161 

.36411 

.32266 

.29076 

.26512 

.24386 

.22578 

.21012 

64 

.67089 

.50962 

.42193 

.36473 

.32345 

.29170 

.26619 

.24504 

.22708 

.21153 

66 

.67073 

.50979 

.42233 

.36531 

.32418 

.29257 

.26718 

.24614 

.22829 

.21284 

68 

.67069 

.50995 

.42270 

.36585 

.32487 

.29338 

.26811 

.24717 

.22941 

.21405 

62 

.67033 

.51022 

.42337 : 

.36682 

.32609 

.29484 

.26977 

.24903 

.23144 

.21624 

66 

.67010 

.51048 

.42395 

.36767 

.32718 

.29612 

.27123 

.25065 

.23322 

.21817 

70 

.66990 

.51069 

.42447 

.36843 

.32813 

.29725 

.27252 

.25209 

.23479 

.21987 

74 

.66972 

.51089 

.42492 

.36910 

.32898 

.29826 

.27368 

.26237 

.23619 

.22138 

78 

.66957 

.51107 

.42534 

.36970 

.32975 

.29917 

.27471 

.25452 

.23745 

.22274 

82 

.66942 

.51122 

.42571 

.37024 

.33043 

29998 

.27564 

.25556 

.23859 

.22397 

90 

.66917 1 

.61150 

42636 

.37118 

.33162 

.30139 

.27725 

.26736 

.24055 

.22609 

98 

.66897 

.51172 

.42689 

.37197 

.33262 

.30257 

.27860 

.25885 

.24219 

.22786 

106 

.66879 

.61192 

.42735 

.37263 

.33346 

.30367 

.27974 

.26012 

.24358 

.22936 

114 

.66864 

.51208 

.42774 

.37321 

.33418 

.30443 

.28071 

.26121 

.24477 

.23065 

122 

.66851 

.51223 

.42808 

.37370 

33482 

.30518 

28156 

.26216 

.24681 

.23177 

138 

.66829 

.51247 

.42864 

.37452 

.33585 

.30641 

.28297 

.26372 

.24752 

.23362 

164 

.66812 

.51266 

.42909 

.37517 

.33667 

.30738 

.28408 

.26496 

.24887 

.23508 

170 

.66798 

.51281 

.42944 

.37570 

.33734 

.30817 

28498 

.26596 

.24997 

.23627 

202 

66777 

.51304 

43000 

.37649 

.33836 

.30937 

.28635 

.26749 

.25164 

.23808 

234 

.66762 

.51322 

.43040 

.37708 

.33909 

.31025 1 

.28735 

.26860 

.25285 

,23939 

266 

.66751 

.61335 

.43070 

.37752 

.33965 

.31091 

.28810 : 

.26944 

.25377 

.24038 

330 

.66734 

.51353 

.43112 

.37814 

.34044 

.31185 

.28917 

.27063 

.25508 

.24179 

394 

.66723 

.51366 

.43141 

.37866 

.34097 

.31248 

.28990 

.27143 

.25596 

1 

.24274 

522 

.66709 

.51381 

.43178 

.37910 

.34164 

.31327 

.29081 

27244 

.25707 

.24394 

778 

.66696 

.51396 

.43215 

.37963 

.34233 

.31408 

.29173 1 

.27347 

.25819 

.24516 

1290 

.66684 

.51409 

.43246 

.38007 

.34288 

.31474 

.29248 

.27430 

.25910 

.24613 

2314 

.66676 

.51418 

.43264 

.38036 

.34325 

.31518 

.29298 

.27486 

.25971 

.24680 

00 

.66667 

.61429 

.43290 

.38073 

.34372 

.31573 

.29361 

.27666 

.26048 

.24763 
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error should be estimated from the root mean square successive differences as 
follows: 


P.E, = .6745 




n—1 

23 (®H-i - a:.)* 
-1 _ ■ 

2(n — 1) 


In 1940, J. von Neumann and R. H. Kent in [2] investigated further the estima¬ 
tion of probable error from mean square successive differences (sums of squares 
of first differences). J. von Neumann, R. 11. Kent, H R. Beilinson, and B. I, 
Hart [3] considered the distribution of 

^ S (a+1 - X,f 

W- — 1 1-1 


in a paper which appeared in June 1941. J. D. Wilhams [4] obtained the 

moments of = - , where 
8^ 


s’* = ^ Z3 (*. - if, 

n ,-i 

and indicated that the rth moment of ri is equal to the rth moment of 6* divided 
by the rth moment of s*. The distribution of the ratio of the mean square 
successive difference to the variance has been published by J. von Neumann 
[5], [6] and B. I. Hart tabulated the probability integral and obtained percentage 
points for this statistic ([7], [8]). Indeed, it should be remarked that the statis¬ 
tical theory of successive differences is allied with the problem of serial correla¬ 
tion [9]. Finally, the use of squared differences of higher order than the first for 
estimating variance appears to have been suggested by A. A. Bennett. Quite 
independently, a treatment of the subject was given by Morse [10] in connection 
with problems on exterior ballistics. Various results on successive-difference 
estimation including significance tests have been given by Tintner [13]. One of 
Tintner^s tests involves the use of selected sets of differences. 

3. Definitions and notations. Suppose the observations xi, X 2 , x*, • • • Xn 
are made at times o = ••• < tn = b and the U are uniformly spaced 

without error. Let f{U) be the true trend so that 17, = f{U) is the mean of the 
population from which x» is drawn and €, = x, — rj, is a random error. Further, 
let p be a non-negative integer less than n and denote to the fth backward differ¬ 
ence of order p of x by A^x,, i.e. 

A'®. = A'-^x, - A^‘x._i = i (-l)'(^) 

/m\ _ ml ^ 

\n/ nl(m — n)l * 


where 


and f > p + 1. 
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We define the following: 


(1) 
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(A^OCA”*. 


r 

f) («- 

‘ p) 

<-p+l 



By Eiu) we will mean the expected value of u, whereas the variance of u will 
be denoted by 

Var (u) = E\u - £(«))*. 

Basically, we shall assume that the «, are sufficiently Gaussian and inde¬ 
pendent that 

£(*,) = £(* 5 ) = 0 . 

M, = E{A) = 3<t‘ , 

= £(*:)£(«?), 

whenever i, j, a and 0 are positive integers for which 

i j, I < i < n, I < j < n. 

4. Expected values. We will now determine the mean or expected values 
of ^n,p ^nd dfi,p • 


E{6\,p) = \ 

if) (” - »> 





or 

( 6 ) £(0 = < r ’. 

(see Lemma 1.3 of section 6 below), 



200 


ANTHONY P. MORSE AND PRANK B. GRUBBS 


Continuing, we have 


» 

E(dlp) 


e{ t, (A'ft + AV)*}, 


Cp ) - P ) 


or 


(6) Eidlj = <r* + v*n.p . 

Consequently, we observe, d\,p is on the average larger than cr* by the quantity 
v\,p . In a particular problem, therefore, we are faced with the situation of 
choosing that combination of n and p which (i) regulates the size of v\,p and (w) 
gives the desired precision of our estimate of variance. 

5. The magnitude of v\,p . In order to study the size of v\,p , we will derive 
for this quantity an upper bound which will indicate the applicability of the 
method of differences to non-polynomial as well as polynomial trends. 

Now, 

A” Vi = A'/Cfi) = j f f /'"’(j/i - yt - ••• -Vp) dyp dy^i ••• dyi, 

where U — = h, by straightforward integration. It will be convenient to 

change the order of integration; thus 

A'/Oi) = f ••• f f - yt - • • • - yp) dyi dy, dyt. 

Jo Jo Jti^i 


Since, from Schwarz^s inequality it is clear that 



(13 - a) f {^7(s)l 


3 


ds 


whenever a and are real numbers and g is integrable, we have 

{A%,1* < r /* ■ • • /* f‘' (/'’’>(pi -yt - dyt dy^--- dy,. 

Jo Jo Jtt-i 


Also, 


2 f f f - ^2 - • • • - Vv)]'^ dyi dyp ••• dyt . 

<-p+l •'0 Jo Jtp 

But for 0 < r < (p —* \)h = — a we have 

r - r)}‘dyt = {f^\8)\*da< f {/«■”(«)}*rf*. 

•'tp J ip-^f Jfg 
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Consequently 

Z (A”./*)’ < A' f* . • • [* f‘ {/‘’’*(8)|’ da dy, " dy't = ft””* f {/'”(«))’ da. 

•-•p+l •'0 •'0 Ja Ja 

Since ft = ^-?, we have finally 

n — 1 


(7) 



{/‘'’(s)r da 
b - a ’ 


which is an upper bound for v\,p in terms of the average value of the square of 
the pth derivative of the trend function/. 

If the trend function / is of the polynomial form, 


/(O = 

r-0 


then the effect of the trend can be eliminated from our observations by estimating 
dispersion from (p + l)st differences. However, if it is known that the trend is 
of polynomial form, then an estimate of dispersion based on least squares would, 
of course, be better. In fact, it ^vill be shown later that the precision of 5n.p 
decreases markedly as p increases. The use of d\,p as an estimate of is pri¬ 
marily of value when the type of trend is unknown; however, even when the type 
of trend is known the computational simplicity of d\,p may offset to some extent 
its lack of optimum precision. 

Ijet us reflect on the magnitude of v\,p over a single period of a sinusoidal trend, 
say/(0 = sin t. In (7) we set a = 0, 6 = 27r and secure 


2 

Vn.p 


< 


^ / 27r 

(^)(n - p)V" - V 


Taking n to be the number of observations for a complete period, a tabulation of 
the upper bound for v\,p for this case is given in Table II. Thus, when there 
are about seven or more observations in each interval of length one period, esti¬ 
mation of dispersion from higher order differences may prove of considerable 
value even for this rather extreme type of trend. 

6. Some combinatorial relations. Although we will ultimately establish 
expressions for the variances of s\,p and dn.p , it appears desirable to give first a 
number of combinatorial relations which present themselves in the computation 
of moments. The relations are easily checked and most of them are possibly 
well known. Nevertheless, it will be convenient to record them for reference 
and in some instances to give proofs. In what follows it will be understood that 



= 0 whenever p and q are not such integers that 0 < ^ < p. 
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TABLE II 


\ 

\ 

5 

6 

7 

8 

0 

10 

1 

.617 

.305 

.274 

.201 

.154 

.110 

2 

.676 

.260 

.120 

.063 

.036 

.016 

3 

.751 

.164 

.049 

.018 

.008 

.002 

4 

.106 

.111 

.021 

.005 

.002 

.0003 

5 

— 

.098 

.009 

.002 

.0004 

.0000 


Lemma 1.1. 9(j) = p(5ll)- 
Lemma 1.2. Q = (^ ^ ^). 

Lemma 1.3. E (^) (^ ,) = ^). 


Proof: 


Hence 


and 


Z (^) X* = (1 + xr = {(1 + xYf = 

-??(')(.!>•• 




C?.) - ?C)C/. -- ?C)C^) - ?(4 

Lemma 1.4. J/ p’ + r* > 0 then 7 — l)' 

Lemma 1.5. (p - 2r) 7 ^) “ (f - l))‘ 

Proof: Multiply, using.1.4 and 1.6. 

‘ ’{pX r)’ - ”{(?-,)’ - C - r~ l)}- 


• Major A. A. Bennett communicated this Lemma. 
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Proof: (s - 20 ( 5 ) = a|(* ^ - (® _ }) jfrom 1 . 6 . 

Put 8 = 2p, t ■=^ p — Ty then 

*(»? )' - - r)' - (p- 7 - 1 )}- 

Lemma 1.8. If f is a funcliotiy i, n, p are integers and p + 1 < t < n, then 

Proof: 

Lemm.a. 1.9. // — <»< A(r, s) = A(s, r) < oo for each integer r and s, then 

Z) Ml", = (iU4 — 3<r^) 2 A{r, rf 

+ <r* (e Air, r))' 4- 2/ E E Air, s)\ 

l,r.l j r-1 «•! 

Proof: Let Ar(r, s) = 1 when r < s and let iV'(r, s) = 0 otherwise. Clearly 
E 23 -ACn s)«r<. = ^ Air, r)el + 2 E 23 Nir, s)Air, «)«,«, , 

raal a*l r««l r««l »*»1 


and 


£({|: t Air, s)ere)l^ = £({E Air, r)eif) 

+ 4£ (|E g Nir, 8)Air, . 


Now 


and 


E (|E Air, r)*:!*) = (m4 - c) E Air, r)* + ,r* |e Air, r)J, 
4B ^|E g Nir, a)Air, 


4o‘ 23 23 Nir, 8)4 (r, s)’ 

r.l «-l 

2<r" E E A(r, 8)* - 2<r* E A(r, r)* 

r—1 ««■! r—l 
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The last three relations combine to yield the desired result. 


Lemma 1.10. 


(n - p?E{d\,p) 




Proof: Helped by 1.8, check that 




Therefore 




and apply 1.9 to complete the proof. 
Lemma 1.11. 




Proof. 


s site-)(.-)}■ 

= X ,-t s S (O' - i) C G -.) ■ 

' <-t S S (0 G+j -i) (0 G+? - i)' "^* *■* **^' 
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- ik. {? (0 C+i -1 - .t C +/-1 

"X 

X. C+•■)’ ■ '^ S ” {(T - ■ C - r'-1)}. '“to* i 

X C ? r)’ - ^ {{^k ')' - » - >)} 

- c ? .y - ^ C” k ')’+^ «*)’ • 


Lemma 1.12. 


Proof. 


sxc^y=<"-'’>(?)■ 


E E (•! y= E E(.^ y= E from 1.8; 

r»«l »—ip+l i—p+l r»l »—p+1 raaO ) 

- C - I.) 2 (f)‘- <n - rt (^) 


7. The variances of d^n .p and d\ ,p . In order to get some idea as to the efficiency 
of the statistics d\^p and d\,p , we will examine their variances. We have 

(?)* = Cp)* “ t£(aV,)l*l 

= (^) (n - P)V* + 2(n - p)a‘ 2 ^ ,) - W 

+ 4pa«(2p-iy 
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with the aid of Lemmas 1.10, 1.11, 1.12 and using the relation — Sc* = 0. 
Thus, 


( 8 ) 




=»<” - S. C^)’ - C'; ')■+C”: ')’■ 

If 2p < n, then 

'ff ( 2p Yy{ 2p V^y (2p\ ^ (4p\ 

, \p + r) ^\j> + rj ^\t) \2p)- 

Moreover, ^ = 0. 

Therefore, 

(9) (n - p)* Var (fil.,) = 2(n - “ 4p^^^ ~ <t* 

when 2p < n. 

As for the variance of (fn,p , wc have 

Var (dl.p) = E\d\., - vVp - = E{6U + K., + 

^E{Sl.^-a^) + k„.,\\ 


or 

(10) Var {d\j = Var («.„)“ + E{k\.,), 

since B[(6*,,p — = 0. 

However, from Schwarz’s inequality, it is guaranteed that 

E{k\.^) < 4i>\y. 


Thus 

(11) Var d\,p < Var {b\.p) + 4^*.,^. 


An upper bound has already been given for v\,p in section 5 above. 

8. The efficiency of i* . It is appropriate to consider the efficiency (as 
defined by Fisher [11]) of the statistic s\,p. In this sense, the efficiency of 
5*,., is pven by 


ir(n,p) 


Var si 

VariVp’ 


where »\ 


13 (.Xj - «)* 


n - 1 
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Accordingly, 


TF(n, p) 


2a^ 

(n-l)Var(3U) 


or 


W(n, p) 


( 12 ) _ 

(»- 1 ) {(»- p) (p?,)' - 2? 7+^ ')) 

If 2p < n 



Formulas (12) and (13) were used in preparing Table I given at the end of the 
paper. For convenience in using formulas (1) and (2) the binomial coefficients 



for 0 < p < 10 are given in Table IIL 


If n > 2, then 


(16) 


Win, 1) 


2 1 ^ 2(n - 1) 

3\ 1 3n - 4 ' 

3n - 3 


as was pointed out by von Neumann, Kent, Beilinson, and Hart in [3]. 
If n > 4, then 


(16) “ 36 


{^+n-2}{^"36(n-2)} 


18(n ~ 2)« 

(n - l)(36n - 88)' 
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As a limiting value for n, we have 


(17) 


W{ , p) = Lim W(n, p) = 

n-^oo 



Using Stirling’s formula for the approximation to the factorial, we have 
Lim \/pW"(“, P) = a/-- 

p—*oo f IT 

Thus,asp—> 00 , 1 ^( 00 ,^) tends to zero and is asymptotically equal to 



TABLE III 
The Binomial Coefficient 


p 

(T 

0 

1 

1 

2 

2 

6 

3 

20 

4 

70 

5 

252 

6 

924 

7 

3432 

8 

12870 

9 

48020 

10 

184750 


For the case n > 2, p > 1 and / constant, then = —— -- and « 

n — 1 

and are all unbiased estimates of the population variance Moreover, 
for this case 


= = Var {s\ 

Var (5l.p) Var 


Using sit based on m — 1 degrees of freedom and keeping the trend, /, con¬ 
stant, then m and n may be chosen so that approximately 


Vex (si) = Var(dl.,) 


and for a normal population this means that 


rM = 1 + (n — l)lV(n, p). 
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Using Table I, it may be seen that for constant trend, /, the worth of djo.io as 
an estimate of a for a normal population is about the same as that of sh , whereas 
that of dlo, 1 is about equivalent to sIq . However, if the trend / is not constant 
then the worth of as an estimate of tr* is diminished while that of d\,p is 
increased. 

Similarly, if the trend is cubic over 20 observations then least squares gives 
an unbiased estimate of based on 16 degrees of freedom, whereas ^ 20,4 gives an 
estimate equivalent in precision to about 6.4 degrees of freedom. However, if 
only eight observations follow a cubic trend, then least squares furnish an un¬ 
biased estimate of based on four degrees of freedom whereas dl,i furnishes an 
estimate equivalent to about 1.9 degrees of freedom. Thus, in the case of 20 
observations, cubic least squares is, so to speak, 2.5 times as valuable as ^ 20 . 4 ; 
in the case of eight observations, cubic least squares is 2.1 times as valuable 
as dg, 4 . 

It might be mentioned that the method of differences is of value in estimating 
goodness of fit. If the fit is good, then our estimate of cr* derived from least 
squares should on the average be equal to the estimate derived from a suitable 
d\,p . If the fit is poor then d\,p will be smaller on the average than the former. 

9. The approximate probable error in estimating a from differences. The 
approximate standard error of 8n,p is given by the relation 


S.E. (8n.p) 


1 S.E. (8\ p) ^ _ 

2 <r \/2(n — l)W(ny p) ’ 


If p has been so chosen that p\,p is suitably small then [see equation (11)] 
some confidence may be put in the approximate formulas: 


(18) 

(19) 


S.E (d„.,) p) 

_. 6745^ _ 

P.E. (d„.p) - . 


Formula (19) was used in preparing Table IV which gives the approximate 
probable error to be feared in using dn,p as an estimate of <r. This table should 
yield interesting information whenever p has been chosen so that d\,p is a suitably 
unbiased estimate of 


10. Remarks. We have presented a useful technique for estimating variance 
from higher order differences and have given the precision of our estimate. The 
method of estimating variance from higher order differences appears to be quite 
valuable in cases where the type of trend in our observations is unknown. A 
considerable field of work remains concerning a complete investigation of the 
distribution and other properties of the statistic d\,p . In this connection, 

n 

Baer [12] has already published a study on the stochastic limit of-- . 

Tt 1 

It is hoped that others will contribute to the problem of estimating dispersion 
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TABLE IV 


The Probable Error In Estimating v From Differences* 


X 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 1 

.4769 











2 

.3373 

.4769 










3 

.2753 

.3771 

.4769 









4 

.2384 

.3180 

.4054 

.4769 








d 

.2133 

.2796 

.3495 

.4215 

.4769 







6 1 

.1948 

.2524 

.3104 

.3704 

.4404 

.4769 






7 ■ 

.1803 

.2317 

.2817 

.3318 

.3855 

.4390 

.4769 





8 

.1686 

.2154 

.2596 

.3024 

.3477 

.3969 

.4442 

.4769 




9 1 

.1589 

.2022 

.2420 

.2794 

.3183 

.3604 

.4057 

.4481 

.4769 



10 

.1508 

.1911 

.2274 

.2610 

.2948 

.3311 

.3708 

.4128 

.4513 

.4769 


11 

.1438 

.1816 

.2153 

.2457 

.2758 

.3074 

.3417 

.3794 

.4186 

.4537 

.4769 

12 

.1376 

.1734 

.2048 

.2328 

.2599 

.2880 

.3180 

.3508 

..3867 

.4234 

.4558 

13 

.1323 

.1663 

.1958 

.2217 

.2465 

.2717 

.2983 

.3272 

.3587 

.3930 

.4276 

14 j 

.1274 

.1599 

.1878 

.2120 

.2350 

.2579 

.2818 

.3073 

..3.351 

.3656 

.3984 

15 

.1231 

.1542 

.1808 

.2035 

.2248 

.2459 

.2677 

.2905 

.3152 

.3423 

.3718 

16 

.1192 

.1491 

.1744 

,1960 

.2159 

.2355 

.2554 

.2761 

.2983 

.3223 

.3485 

17 

.1156 

.1445 

.1687 

.1892 

.2080 

.2262 

.2447 

.2637 

.2837 

1 .3052 

.3286 

18 ' 

.1124 

.1403 

.1636 

.1831 

.2009 

.2180 

.2352 

.2527 

.2710 

I .2905 

.3116 

19 

.1094 

,1364 

.1589 

.1775 

.1945 

.2106 

.2267 

.2430 

.2599 

.2777 

.2967 

20 1 

.1066 

.1328 

.1545 

.1724 

.1886 

.2040 

.2191 

.2343 

.2500 

.2663 

.2837 

21 

.1040 

.1295 

.1505 

.1677 

.1832 

.1978 

.2121 

.2264 

.2411 

.2562 

.2722 

22 

.1016 

.1265 

,1468 

.1634 

,1783 

.1922 

.2058 

1 .2193 

.2331 

.2472 

.2620 

23 

.0994 

.1236 

.1433 

.1591 

.1738 

.1871 

.2000 

.2129 

.2258 

.2391 

.2529 

24 

.0973 

.1209 

,1401 

.1557 

.1695 

.1824 

.1948 

.2069 

.2191 

.2316 

.2446 

25 

.0954 

.1184 

.1371 

.1522 

.1656 

.1779 

.1898 

.2015 

.2131 

.2249 

.2370 

26 

.0935 

.1160 

.1343 

.1490 

.1619 

.1739 

.1853 

.1964 

.2075 

.2187 

.2301 

27 

.0918 

.1138 

,1316 

.1459 

.1585 

.1700 

.1810 

.1917 

.2023 

.2130 

.2238 

28 

.0902 

.1117 

.1291 

.1431 

.1553 

.1664 

.1770 

.1873 

.1975 

.2077 

.2180 

29 

.0885 

.1097 

.1268 

.1404 

,1522 

.1631 

.1733 

1832 

.1930 

.2028 

.2126 

30 

.0871 

.1078 

. 1245 

.1378 

.1493 

.1599 

.1698 

.1794 

.1888 

.1981 

.2076 

31 

.0857 

.1060 

.1224 

.1354 

.1466 

.1569 

.1665 

.1758 

.1848 

.1938 

.2029 

32 

.0843 

.1043 

.1204 

.1331 

.1441 

.1540 i 

.1634 

.1724 

.1811 

.1898 

.1985 

33 

.0831 

.1027 

.1184 

.1309 

.1416 

.1514 

.1605 

.1692 

.1776 

.1860 

.1944 

34 

.0818 

.1012 

.1166 

.1288 

.1393 

.1488 

.1577 

.1661 

.1744 

.1825 

.1905 

35 

.0807 

.0999 

.1149 

.1268 

.1371 

.1464 

.1550 

.1632 

.1713 

.1791 

.1869 

36 

.0795 

.0983 

.1132 

.1249 

.1350 

.1441 

.1525 

.1605 

.1683 

.1759 

.1834 

37 

.0784 

.0969 

.1116 

.1231 

.1330 

.1418 

.1601 

.1579 

.1655 

.1729 

.1802 

38 

.0774 

.0956 

.1101 

.1214 

.1311 

.1397 

.1478 

.1555 

.1628 

.1700 

.1771 

39 

.0764 

.0943 

.1086 

.1197 

.1292 

.1377 

.1456 

.1531 

.1603 

.1673 

.1741 

40 

.0754 

.0931 

.1072 

.1181 

,1274 

.1358 

.1435 

.1508 

.1578 

.1646 

.1713 



ESTIMATION OP DISPERSION 


211 


TABLE IV —Continued 


XP 
n \ 

\ 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

42 

.0736 

.0909 

.1045 

.1151 

.1241 

.1322 

.1396 

.1466 

.1533 

.1597 

.1661 

44 

.0719 

.0887 

.1020 

.1123 

.1211 

.1288 

.1360 

.1427 

.1491 

.1553 

.1613 

46 

.0703 

.0868 

.0997 

.1097 

.1182 

.1257 

.1326 

.1391 

.1453 

.1512 

.1670 

48 

.0689 

.0849 

.0975 

.1073 

.1155 

.1228 

.1295 

.1357 

.1417 

.1474 

.1529 

60 

.0675 

.0832 

.0955 

.1050 

.1130 

.1201 

.1266 

.1326 

.1383 

.1438 

.1492 

52 

.0661 

.0815 

.0936 

.1029 

.1107 

.1176 

.1238 

.1297 

.1352 

.1405 

.1467 

54 

.0649 

.0800 

.0918 

.1009 

.1085 

.1152 

i .1213 

.1270 

.1323 

.1375 

.1425 

66 

.0637 

.0785 

.0901 

.0990 

.1064 

.1129 

.1189 

.1244 

.1296 

.1346 

.1394 

58 

.0626 

.0771 

.0885 

.0972 

.1045 

.1108 

.1166 

.1220 

.1271 

.1319 

.1366 

62 

.0606 

.0746 

.0855 

.0939 

.1008 

.1069 

.1125 

.1176 

.1224 

.1270 

.1313 

66 

.0587 

.0723 

.0828 

.0909 

.0975 

.1034 

.1087 

.1136 

.1182 

.1225 

.1266 

70 

.0570 

.0702 

.0804 

.0881 

.0946 

.1002 

.1053 

.1100 

1 .1144 

.1185 

.1224 

74 

.0554 

.0682 

.0781 

.0856 

.0919 

.0973 

.1022 

.1067 

.1109 

.1149 

.1186 

78 

.0540 

.0664 

.0760 

.0833 

.0894 

.0947 

.0994 

.1037 

.1077 

.1115 

.1152 

82 

.0527 , 

.0648 

.0741 

.0812 

.0871 

.0922 

.0968 

.1009 

.1048 

.1085 

.1120 

90 

.0503 

.0618 

.0707 

.0774 

.0830 

.0878 

.0921 

.0960 

.0997 

.1031 

.1063 

98 j 

.0482 

.0592 

.0677 

.0741 

.0794 

.0840 

.0880 

.0917 

.0952 

.0984 

.1014 

106 

.0463 

.0569 

.0650 

.0712 

.0762 

.0806 

.0845 

.0880 

.0913 

.0943 

.0972 

114 

.0447 

.0549 

.0627 

.0686 

.0734 

.0776 

.0813 

.0847 

.0878 

.0907 

.0934 

122 

.0432 

.0530 

.0606 j 

.0663 

.0709 

.0749 

.0785 

.0817 

.0847 

.0875 

.0900 

138 

.0406 

.0498 

.0569 

.0622 

.0666 

.0703 

.0736 

.0766 

.0794 

.0819 

.0843 

154 

.0384 

.0472 

.0538 

.0589 

.0630 

.0664 

.0695 

.0723 

.0749 

.0773 

.0795 

170 

.0366 

.0449 

.0512 

.0560 

.0599 

.0632 

.0661 

.0687 

.0711 

.0734! 

.0765 

202 

.0336 

.0412 

.0470 

.0513 

.0548 

.0578 

.0605 

.0629 

.0650 

.0671 

.0689 

234 

.0312 

.0382 

.0436 

.0476 

.0509 

.0537 

.0561 

.0583 

.0603 

.0621 

.0639 

266 

.0292 

.0359 

.0409 

.0446 

.0477 

.0503 

.0525 

.0546 

.0565 

.0582 

.0698 

330 

.0262 

.0322 

.0367 

.0400 

.0428 

.0451 

.0471 

.0489 

.0505 

.0521 

.0635 

394 

.0240 

.0295 

.0336 

.0366 

.0391 

.0412 

.0430 

.0447 

.0462 

.0475 

.0488 

522 

.0209 

.0256 

.0292 

.0318 

.0339 

.0357 

.0373 

.0387 

.0400 

.0412 

.0423 

778 

.0171 

.0210 

.0239 

.0260 

.0278 

.0292 

.0305 

.0317 

.0327 

.0337 

.0346 

1290 

.0133 

.0163 

.0185 

.0202 

.0216 

.0227 

.0237 

.0246 

,0254 

.0261 

.0268 

2314 

.0099 

.0121 

.0138 

.0151 

.0161 

.0169 

.0177 

.0183 

.0189 

.0196 

.0200 


* If dn*p is a sufficiently unbiased estimate of then the approximate probable error 
to be feared in using dn,p as an estimate of <r may be obtained by multiplying the following 
tabular entries by a. 
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when observed data display trends as it is believed that the method of differences 
deserves much attention. In particular, it is hoped that someone will have the 
time and ingenuity to calculate the distribution of the statistic 





Were this done, an admirable criterion would be at hand for gauging the signifi¬ 
cance of a change in the estimate of a as we pass from differences of order p to 
those of order p + 1. Of course, useful information in this connection could be 
obtained from a knowledge of the distributions of and 5«,p+i ; in fact their 
variances as herein calculated give us a basis for somewhat reasonable conclu¬ 
sions. An expression for the standard error of the difference between the 
estimates of <? from two consecutive scries of finite differences is given in 
[13, Chapter VI]. 

In connection with testing goodness of fit, it would be valuable also to know 
the distribution of 

^ * 
dn.p+l 

where is the estimate of variance derived from the least squares fitting of a 
polynomial of degree p. 

For convenience of reference, we conclude the paper with 

11. A concise description of the method and its precision. It frequently 
happens that successive observations made at regular intervals are subject to 
the same standard error a while the means of the populations from which they 
are drawn display a trend. We give here a method of estimating the variance 
and of determining the precision of our estimate. This method is primarily of 
value when the trend is unknown; however even when the type of trend is known, 
its computational simplicity may make the method advantageous. 


The method. Arrange the data in a vertical column and then in the usual 
way form difference columns of order 1, 2, • • , p. Sum the squares of the pth 

order differences and divide by the number (n — p) 

is the number , where 


2p\ 

V/ 


Our estimate of <t 


dl.p 


_ JL_ 

(^)(n - v) 


t 

»-P+l 



(Xt — 2X1+1 X,42)* 


n 


Xi 


* Dixon [91 gives moments of the statistic 


^ (a;,’ - Xi+iY 


where x«+i 
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The precision. The precision of this estimate may be determined from the 
following information (which has been derived in the present paper): 


E{dl,) = cr* + V* 


2 

Vf,,p 


< -1- /6 - a\(b - lf^\8)Yd8 . 

“ /2p\ \n — p/\n — 1/ Ja b — a * 

\P/ 

Var(dn.p) < Var(a*„.p) + 4:v\,pa ; 


Var (6\,p) 

where 17(n, p) is given in Table I. 


2a* 


(n - l)W{n,p)^ 


TABLE V 


p 


Cy 


1 

18.90 

184.62 

11.22 

2 

1.21 

1.88 

10.56 

3 

.88 

1.85 

10.30 

4 

.87 

1.84 

10.12 

6 

.86 

1.83 

10.01 


In case v^^.p is sufficiently small (this is determined by the requirements of the 
given problem), then Table IV may be used directly to determine the approxi¬ 
mate probable error in using dn,p as an estimate of a. 

An example. As a practical example of the use of the method of differences 
when the trend is unknown and of the stability of the statistic d\,p with respect 
to p, we mention a recent problem at Aberdeen Proving Ground which had to do 
with estimating the accuracy with which certain photographic measurements 
locate a moving object. Ballistic Cameras were used to determine horizontal 
X and y, and vertical z coordinates (all in feet) of an airplane traveling about 
160 mph at an elevation of about 35,000 feet. An automatic pilot was in use in 
the airplane as it flew over a three mile course. At one second intervals for a 
period of 70 seconds two Ballistic Cameras, 5000 feet apart, were used to locate 
the plane. Since the plane was traveling pretty much in the y direction one 
would expect; that first differences would yield a standard error in y far in excess 
of its true one; that second differences would furnish a much better estimate; 
and that perhaps third differences would yield a still more trustworthy one. No 
matter what order of difference is used we never expect such an estimate to be 
too small. In this problem, the standard errors in a?, z as estimated from dif¬ 
ferences of certain orders, p, were as given in Table V. 
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THE EFFICIENCY OF SEQUENTIAL ESTIMATES AND WALD'S 
EQUATION FOR SEQUENTIAL PROCESSES 

By J. Wolpowitz 
Columbia University 

1. Summary. Let n successive independent observations be made on the 
same chance variable whose distribution function /(x, B) depends on a single 
parameter B, The number n is a chance variable which depends upon the out¬ 
comes of successive observations; it is precisely defined in the text below. Let 
B*{xi ,• • • , x„) be an estimate of B whose bias is b{B), Subject to certain regu¬ 
larity conditions stated below, it is proved that 

.V) > (i + f 

When /(x, B) is the binomial distribution and B* is unbiased the low^er bound 
given here specializes to one first announced by Girshick [3], obtained under no 
doubt different conditions of regularity. When the chance variable n is a con¬ 
stant the lower bound given above is the same as that obtained in [2], page 480, 
under different conditions of regularity.^ 

Let the parameter B consist of I components , • • • , for which there are 
given the respective unbiased estimates 0f(xi , • • • , Xn), • • • , ,• • • , Xn). 

Let II \ij II be the non-singular covariance matrix of the latter, and || || its 

inverse. The concentration ellipsoid in the space of (ifei , • • • , ki) is defined as 

- ediki - Bi) =1+2. 

(This valuable concept is due to Cram4r). If a unit mass be uniformly dis¬ 
tributed over the concentration ellipsoid, the matrix of its products of inertia 
will coincide with the covariance matrix || X,y ||. In [4] Cramer proves that no 
matter what the unbiased estimates , • • * , , (provided that certain regu¬ 

larity conditions are fulfilled), when n is constant their concentration ellipsoid 
always contains within itself the ellipsoid 

~ — Bj) = I + 2 

•'ll 

where 

_ log A 

__ M, )■ 

^ To whom this result is to be ascribed is not clear from the context in which Professor 
Cram4r describes it (in [2]). After the present paper was completed the author learnedx>f 
the papers by Rao [8] and Aitken and Silverstone [0], both of which deal with this question. 
The author is indebted to Prof. M. S. Bartlett for drawing his attention to these papers. 
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Consider now the sequential procedure of this paper. Let , * * • » be, 
as before, unbiased estimates of , • • • , 0i, respectively, recalling, however, 
that the number n of observations is a chance variable. It is proved that the 
concentration ellipsoid of 6* , sf always contains within itself the ellipsoid 

- (J.) I + 2 

where 




EnE 


(d log /d log A 

k dOi /• 


When n is a constant this becomes Cramer’s result (under different conditions 
of regularity). 

In section 7 is presented a number of results related to the equation 
EZn = EnEXy which is due to Wald [ 6 ] and is fundamental for sequential 
analysis. 


2 . Introduction. I^et X be a chance variable whose distribution function 
f{x, B) depends on the parameter 0. It is assumed that X either has a probability 
density function (which we then denote by /(a:, B)) or that it can take only 
an at most denumerable number of discrete values (in the latter case 
/(x, B) = P{X = x), where the latter symbol denotes the probability of the 
relation in braces). I^et a? = Xi, X 2 , • • • be an infinite sequence of observations 
on X, and let Q be the space of “points^^ w. Let there be given an infinite 
sequence of Borel measurable functions^i(xi),^(xi ,^ 2 ), • • • ,^;(xi, • • • ,x,), • • • 
defined for all w in , such that each takes only the values zero and one. It is 
well known that the function /(x, B) defines a measure (probability) on a Borel 
field in fi. We assume that everywhere in 0 , except possibly on a set whose prob¬ 
ability is zero for all B under consideration, at least one of the functions ^ 1 , ^ , • • • 
takes the value one. Let n(aj) be the smallest integer at which this occurs. 
Thus n(co) is a chance variable. 

In statistical applications the chance variable n(a)) may be interpreted as a 
rule for terminating a sequence of observations on the chance variable X, the 
probability of termination being one, and the decision to terminate depending 
only upon the observations obtained. A sequential test is an example of this 
procedure. The converse is, however, not true, because the process described 
above does not require that any statistical decision should be reached when the 
process of drawing observations is terminated. 

An ^^estimate’’ of 0 is a function 0 *(xi, • • • , x„) of the observations Xi, • • • , x„ 
(those obtained prior to the ‘‘termination*' of the process of drawing observa¬ 
tions) . In the sequel we shall limit ourselves to estimates whose second moments 
are finite. The estimate is “unbiased" if EB*^ the expected value of 0 *, is 0. 
When this is not so P 0 * — 0 is called the bias, 6 ( 0 ), of 0 *. In general the bias 
is a function of 0. It is obvious that the function 0* may be undefined on a set 
of points («i, • • • , x„) whose probability is zero for all 0 under consideration. 
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In the present paper we shall be concerned with an upper bound on the effi¬ 
ciency of a sequential estimate, or, more precisely, with a lower bound on its 
variance. This lower bound is intimately related to certain results on the effi¬ 
ciency of the maximum likelihood estimate from a sample of fixed size. This is 
not surprising since fixed-size sampling is a special instance of sequential sam¬ 
pling. The results obtained in this paper are also obviously and intimately 
related to those due to Cramer [4] and those described by him in [2], pp. 477-488. 
Naturally the conditions of regularity (restrictions on f(x, d), 0*, etc.) under 
which the results are proved are different. For example, no restrictions on the 
sequential sampling procedure need appear in the statement of a theorem which 
deals only with samples of fixed size. 

The argument below proceeds as if f{x, $) were a probability density function. 
The results apply equally well to the case where/(x, 6) is the probability function 
of a discrete chance variable provided: 

1) . Integration is replaced by summation wherever this is obviously required. 

2) . The phrase “almost all points*’ in a Euclidean space of any finite dimen¬ 
sionality is understood 

a) . as all points in the space with the possible exception of a set of Lebesgue 
measure zero, when/(a:*, 6) is a probability density function 

b) . as all points in the space with the possible exception of points one of whose 
coordinates is a member of the set Z, when f(Xy 6) is the probability function of a 
discrete chance variable. The set Z consists of all points z such that/( 2 , 0) = 0 
identically for all $ under consideration. 

3- Conditions of regularity. In this section we shall formulate the restrictions 
which we impose on /, the estimates, and the sequential process. They are 
intended to be such as will be satisfied in most cases of statistical interest. No 
doubt they can be weakened, but the author has decided against attempting to 
do so here. The list may seem long for two reasons. Seldom in the literature 
are the assumptions which, for example, lead to validation of differentiation 
under the integral sign etc., formulated explicitly. The presence of a sequential 
procedure means that additional restrictions must be imposed. 

In this section we assume that 0 is a single parameter. The case where d has 
more than one component is treated later. 

(3.1) . The parameter 6 lies in an open interval D of the real line. D may consist 
of the entire line or of an entire half-line, 

(3.2) . The derivative^ exists for all 6 in D and almost all x. We define 

od 

- as zero whenever f(x, B) = 0; thus is defined for all 6 in D and 

d$ od 

almost all x. We postulate that —- ■ = 0 and that E ^ 

oB \ oB / 

he not zero for all B in D. 
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exists for all Q in D. 

(3.4). Let B^t {j = 1, 2, • • • )» set of points (xi, • • • , Xj) in thej-dimen- 
sional Euclidean space such that 

<Piixi , • • • , a;<) = 0 z = 1, 2, • • • ,y - 1 

y ••• = 1- 

For any integrals there exists a non-negative L-measurable function Tj{xi , • • • , xy) 
svxh that 


a). 


,Xi)^Uf(Zt,0) 

OO 


< T,ixi,■■■ ,Xi) 


for all dinD and almost aU{xi, ••• ,x,)inRj 


b). / Tj(xi , • • • , X,) dxi • • • dxj 

jRf 

is finite. 

(3.5). Let 

m = / . • • •, Xi) ilfix, ,e)dXi, O' = 1, 2, •. • 

We postulate the uniform convergence of the series 

dtj{B) 
i de 


{the existence of 

dd 


is a consequence of Assumption (3.4)) for all B in D. 


4. The case of one parameter. In this section we assume that f{x, 6) depends 
on a single parameter 6. In sections 5 and 6 we shall discuss the case when 0 
is a vector with more than one component. 

Wehave£ll?i^)=0 

de 

by (3.2). Define the chance variable 

^ 3 log f(Xi, 0) 

{A do 

By an argument almost identical with that of [1], Theorem 1, or of Theorem 7.1 
bdow, we have 

(4.1) 


EYn = 0. 
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From Theorem 7.2 below we obtain 

(4.2) Ay,) = EnE 

Let (xi, • • • , Xn) be an estimate of B such that 

EB^ = 6 + h{B). 

Then 

(4.3) S [ 6*(xi, ,Xj) n/(a:<. dxi = e + h{0). 

j-1 JRi t-1 

Differentiation of both members of (4.3) with respect to B (Assumptions (3.4) 
and (3.5)) gives 

(4.4) = 1+^^. 


From (4.1) it follows that (4.4) gives the covariance between 6* and y« . 
from (4.2) 


(4.5) A9*) > (l + ^EnE 


Hence 


When the bias h{B) is constant, for example when h{B) s 0 in case B* is an 
unbiased estimate, we have from (4.5) 


(4.6) 


Ae*) > *• 


The equality sign in (4.6) will hold if B* may be written as Z'{B)Yn + Z”{B), 
where Z' and Z" are functions of B, However, B* itself should not be a function 
of B if our argument is to remain valid. The subject is connected with the 
question of the existence of a sufficient estimate. 

Let/(a;, B) be defined as follows: 

f{x, e) = e*(l - ey-% (x = O or l; O < e < l). 

Then 

d log fix, B) _x (1 - x) log/V _ 1 

dB B (1 - ^) ’ "\ dB J Bil - B)' 

Suppose B* is unbiased. Then cr’(0 > ^(1 — B)iEn)~^, a result first given by 
Girshick [3] under unspecified regularity conditions. 

Let the functions ^ • be such that n(w) is a constant. We are then 

dealing with samples of fixed size. The result (4.5) is then given in [2], p. 480, 
under different conditions of regularity. 


5. Regularity conditions for the case when B has more than one component 
We suppose that B = (^i, * * * , Bi) and that simultaneous estimates 
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0i{xi , • • • ,Xn), • • * , , • •• , a;„) of the components of 6 are under discussion. 

In the sequel we shall limit ourselves to the case when these estimates are all 
unbiased. 

We postulate the following regularity conditions which are suflSicient to validate 
section 6: 

(5.1) . The covariance matrix of the estimates B\ d* is non-singular for all 

6 in D (this time D is an open interval of the l-dimensional parameter space), 

(5.2) . The conditions of section 3 are satisfied for each $i and 0? (t = 1, • • • , Z). 

6. The ellipsoid of concentration when B has more than one component. Let 

B = (Bl , • • • y Bl), 

We shall first describe briefly the result of Cramer [4] which refers to samples 
of fixed size n > I, Let B*{xi, • • • , aj») be an unbiased estimate of 
Biy {i = 1, • • • , Z). Let II Xii II be the non-singular covariance matrix of the 
B* , and let || || be its inverse. The ‘‘ellipsoid of concentration** in the space 

of points (Ai, • • • , ki) is defined as 

(6.1) E - ediki - ed = 1 + 2. 

i.j-1 

If a unit mass be distributed uniformly over this ellipsoid it will have the point 
(^ 1 , • • • , 0i) as its center of gravity and X<, as its product of inertia about the 
corresponding axes, Cramer proves that, subject to certain regularity condi¬ 
tions, there is a fixed ellipsoid 

I 

(6.2) E - edih -di) = 1 + 2 

where 

which is always contained entirely within the concentration ellipsoid of any set 
of unbiased estimates. The tw^o ellipsoids coincide only under certain condi¬ 
tions, among which is that the Bf be jointly sufficient estimates of the Bi . 

I^et us now consider the sequential procedure of this paper and postulate the 
regularity conditions of .section 5. Let 

K = II A,,-II 

be a matrix with real elements such that | | = 1 and let 

be its inverse. Let 



fll 

91* 


ll^ll = 

• , 119*11 = 

• , 111^11 = 

• 


9i 

9r 
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be column matrices. Suppose 

(6.3) II ^II = KII ell. 

Then 

(6.4) llflil = *:-Mli^ll. 

Define 

lAi* 

• =^c||e*l|. 

From section 4 we have 

(6.5) EnE > [/(^r)]- 

where the differentiation by which ^.^^-'^is obtained is performed with , • • •; 

d\f/i 

held constant. Consider the last (I — 1) rows of K as fixed and (kn , fci 2 , • • • , ku) 
as free to vary subject only to the restriction that \ K \ = 1. The left member 
of (6.5) is then a fixed quantity, while the right member is a function of the first 
rovT of K.. The inequality (6.5) must remain valid for all admissible 
(A;ii, ‘ ‘ , ku) . Hence (6.5) will remain valid if the right member of (6.5) is 
replaced by its maximum with respect to {kn , • • • , ku) . We shall obtain this 
maximum and find that (6.5) then implies a result about the minimal ellipsoid 
of concentration. 

The problem is therefore to minimize Now 

(6.6) <T*(^r) = 23 Xiy/Jiifciy. 

».y 

The family of ellipsoids in the space of (Aju , • • • , ku) 

(6.7) 23 ^iikiikij = c, 

i,i 

where c is a running parameter, has all centers located at the origin. Let 

(fcfi , • • • , /bi?) 

be the sought-for maximizing values of (A:ii, • • • , ku). From the definitions of 
K and we have 

(6.8) 

where (A;^', • • • , are constants. It follows that the minimum value 

Co of (r*(^f) is such that the ellipsoid 

(6.9) 23 K'hifcii = Co 

iti 

is tangent to the hyperplane (6.8) at the point (fcJi, • • • , fcii). Now the tan¬ 
gent plane to (6.9) at this point is given by 

(6.10) A/itA»i/ =* Co. 
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From (6.8) and (6.10) we obtain 


(6.11) 

= E k\i\ii, 

i 

O' = 1, • ■ 

■•,0. 

Hence 




(6.12) 

Co E X’' A:* ‘ = kl, 

i 

(/ = 1, • 

■■,1) 

from which 




(6.13) 

Co E y’k*^k’^ = 1. 

*.y 



We have 




(6.14) 

3 log/ 

i dOi 

(a log/V _ v ,jx a ic«/a log/ 

\ ati>x ) h dBi aoi • 




From (6.5), (6.13), (6.14), and the definition of co we conclude that 

(6.15) E Mi; fc"fc'* > Z X'' Jfc“ k’'^ 

i,l t,i 


where 

(6.16) 

We may restate (6,15) as follows: The concentration ellipsoid 

(6.17) Z X”(A;.- - e.)(fcy -Oi) = 1 + 2 

ij 

of the unbiased estimates , • • • , always contains within itself the ellipsoid 

(6.18) Z - OiKk,- -Si) = 1 + 2 

where the mo* are defined by (6.16), 

The question of the coincidence of the two ellipsoids is connected with the 
question of the existence of sufficient estimates. It may be difficult to state 
any general results about the concentration ellipsoid of biased estimates without 
postulating some relationships among the biases and/or their derivatives. 

7. On Wald’s equation and related results in sequential analysis. In sec¬ 
tion 4 we referred to a proof by Blackwell [1] of an equation due to Wald [5] 
which is fundamental in the Wald theory of sequential tests of statistical hypothe¬ 
ses. Here we shall give a perhaps simpler proof of this equation, and then prove 
several new and related results of general interest for sequential analysis. 

The results of Theorems 7.2 and 7.3 below can be obtained by differentiation 
of Wald’s fundamental identity of sequential analysis ([6], [7]). However, the 
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conditions under which we obtain these results are less stringent than any so far 
found sufficient to establish the identity and the validity of differentiating it. 
Theorem 7.4 and its corollaries refer to sequential processes where the chance 
variables may have different distributions or even be dependent. In the future 
we hope to return to the question of finding all central moments of Zn, the 
problem of generalizing the fundamental identity, and related questions. 

For Theorems 7.1, 7.2, and 7.3 we shall assume a^chance variable X whose 
cumulative distribution function F(x) is subject only to whatever restrictions 
may be explicitly imposed on it in each theorem. We assume the existence of a 
general sequential process such as is described above, which is subject only to 
such restrictions as may be explicitly formulated in each theorem. The sequen¬ 
tial process of course defines the chance variable n. I^t Xi, X 2 f • • • be suc- 

n 

cessive independent observations on X. We define Zn — ^ x,. If E{X) and 

<r*(X) exist we shall denote them by w and , respectively. 

Theorem 7.1 (Wald [5], Blackwell [1]) . Suppose w and En exist Then 

(7.1) E(Zn — nw) = 0. 

The following theorem, which is a sort of partial converse of Theorem 7.1, is 
proved concomitantly with Theorem 7.1: 

Theorem 7.1.1. If EZn exists, and if either P{X > 0} ~ OorPfX < Oj = 0, 
then w and En both exist, and 

EZn = wEn, 

Actually the same proof suffices for a somewhat stronger form of Theo¬ 
rem 7.1.1; 

Theorem 7.1.2. If EZn exists, and if 

EiXi I n = i) > 0 (or < 0) 

for all positive integral j such that P {n = j] 0, and all i < j, then w and En 
both exist, and 

EZn = wEn . 

n \2 

I a:,* — to 1 I exists, then and En both exist, and 

f-i / 

(7.2) E{Zn — nwf = <r^En . 

We have 

E{Zn - nw) = E (xi - w)^ (zi - «>)) n dF(x,) 

\im,l / y-1 JRf \i-l / <.1 

i-1 M JRi m-1 


Theorem 7.2. If E 


( 


(7.3) 
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(7-4) £ f (Xi - w) n dF(xJ = P{n> j}E(Xi - «;) = 0. 

imij •^Ri maal 


Hence 


tit (»,-»)n«•(..)-0. 

j—I *—y •fRi m-al 


From this (7.1) follows. 

Suppose now that the conditions of Theorem 7.2 are fulfilled. We have 

E{Z. - nu>y = t. f (i. (A - w))' n dFM 

j-1 JRi \t-l / m-l 

(7.6) = t. 11 f (x,- w)^ n dFM 

J—1 t—y *fRi m.-.! 


+ 2i'ttf (x. - w){xj - w) n dF{Xm). 
y—2 «-it immj JRi 


I^et s < j be any two positive integers. Then 


Hence 


f - w) n dF{x„) = 0. 

y R i ffiwl 

tzif (x. - w)(xi - w) n dF(x„) = 0. 
y—2 1 i—y m—1 


In a similar manner we obtain 

(7.9) E [ (X, - wy n dF(x„) = yp{n > j\. 

immj JRi msMl 

From (7.6), (7.8), and (7.9) it therefore follows that 

(7.10) E(Zn - nwf = <r* £ P[n > j\ = <r* £ jP[n = j\ = <r^En 

y -1 1-1 

which is the desired result. 

It remains to prove the validity of rearranging the series in (7.3) and (7.6). 
First, we have 

(7.11) E f I a:; — to I ri dF(xm) = P{n > j}E j X - to |. 

t-y •fRi m—l 
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Hence it follows that 


(7.12) 


ZE/ |x,-ic|ndF(x«) 

j.l JRi m.1 y.l 


w 


E\X - w\ = ii = B 1 X - w I Pn. 


This justifies the rearrangement of terms in the series in (7.3). Second, the 
series (7.6) is dominated by the series 


(7.13) 


E E / (*,• - W)* ri dP(x«) 

}—1 «a>I *'R{ ftal 


+ 2EEE/ |x. - wl-la;, - wiri dF(aw) 

all of whose terms are positive. The series (7.13) converges because 

(7.14) p I x< - w ly <+«■ 

Hence the rearrangement of the series (7.6) is valid. 

In the sequel we require certain sets RJC; = 1,2, * ) which we shall define 
now. I^et , i < j, be the totality of all points («!, • • • , »,) such that 

(7.15) (ici, • • • , xd € Ri . 

Let R^ be the ^-dimensional Euclidean space. Then 

(7.16) R'l - i, R*i. 

*-.1 

We shall now prove: 

Theorem 7.3. Suppose that E [§ |x, - w| J and En [s |x. - w|J 
exist^ Then 

(7.17) 
where 


E{Zn “ nwY = w^En 4 - Sa^En{Zn — nw) 
W3 = EiX - 


exists. 


* The author has succeeded in proving that the existence of ^ ^ ^ 1 — tr | J implies 

of 2 1 ^ ij* proof will be published subsequently in con- 


the existence 


nection with other results. 
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Proof: We have 


E{Zn - nw)* “ S f - w^)l n dF{xJ 

j-i L<-i J i»-i 

= ]£/' i (**-«»)’rrdF(x») 

j»l *'JE/ «»1 * nusl 

(7.18) + 3 Z f £ E (x. - - «>)’ 11 dF(xJ 

2-3 •'Ay t—2 1 ftta-l 

+ stf tZ (x, — w)*(Xi — -U^) n dFioCm) 

2>.2 •'Ay t«2 «»1 iih-1 

+ 6£f igg (xt — — w)(xi — II?) ri 

jmmZ •'Ay t—8 «—2 <■>! m«-l 

Considering the first term in the right member of (7.18), it follows that 

£/ ri(x.- - «))’lr[dw 

j™l*'AyL*“l J »i«l 

v7.iy; t-i •lAy «-i 

00 

= > i] 

00 

= ^iw^Pln = i] = WzEn. 

All the rearrangements of terms in the operations involved in the proof of Theo¬ 
rem 7.3 are legitimate because the various series are absolutely convergent. 

As for the second term in the right member of (7.18), we have 

t\ tg {Xt — w){Xi — wf n dF{x^) 

jmmi •'Ay t—2 «—1 m—1 

«on\ = Z Z g f (*. - w){xi - u;)*n dF{x„) 

= <r*ZZf (X. - w)n dF(x„) 

*>■1 i—1+1 •? A'y~i m—1 

= «^‘ZZf (X. - w)il dF(xJ. 

$mml immS •'A'| 

We now operate on En{Zn — iwi?), and obtain 

J5n(Z» -nw) = Ef jg (x< - w) U dF(z„) 

(7,21) /-*•>*/ <-» »-> 

ZZ f - w)!! dF(xm). 

i-i i-i Jm{ ■-> 
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We observe that 


(7.22) 


• r * 

/ iixj — w) n dF(x„) 

»—/ ma>l 

= i 2 f (% - w) n 

«-y jRi m.1 

+ 2 S / (®y - w) n 

#—j+l t—# •'Ri m—l 

To evaluate the left member of (7.22), we proceed as follows: It is easy to see that 

(7.23) £ f (x,- - to) U dF(,xJ = 0. 

t-j JRi -1 

Moreover, when s > j, 

(7.24) £ f (x,- -to)U dF(x„) = f (x, - w) E dF(x„). 

tMt *fR{ fflaal —I KImI 

Hence 

(7.25) £ f i (*,• - t«) ri dFM = £ f (xy - ix) n dF(x..). 

immj JRi mrnrnl Mmmj J B* ^ fflail 

Therefore 

(7.26) En{Zn — nw) = ttl. (Xy - W) n dF(x„). 

•-./ Jr[ fm.1 

It remains now to consider the third term of the right member of (7,18). 
We have 


(7.27) 


£ f Z) £ (a:. - w)* (Xi -10)11 dF(x, 

}—2 vKy {..2 «.«1 ftal 


= £££/” (x. - u>)* (Xj - tx) n dF(xJ. 

9wml Umt+l •'Bf m^rnl 

Now, suppose that in the expression 

(7.28) V,if = f (xs - wf (xi - w)Jl dF{x^) 

jRf m—I 

where j > i > », we integrate with respect to all Xm for which m > i. Then 
it is not difficult to see that 

ao 

(7.29) £ F,yy = 0 

for all 8 and i such that 1 < s < t. Hence from (7.27) 

(7.30) £ f io)*(xt - to) dF(xm) -= 0. 

y-t Jr/ imt mrnl 
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In a similar way it is shown that the fourth term of the right member of (7.18) 
is zero. 

The desired result (7.17) is a direct consequence of (7.18), (7.19), (7.20), 
(7.26), and (7.30). 

Consider now an infinite sequence of chance variables Xi, x^, • • • ^ which 
need not have the same distribution and which may be dependent (in which 
case they must satisfy the obvious consistency relationships). We take suc¬ 
cessive observations on these chance variables and define a sequential process 
as above, which is subject only to such restrictions as we shall explicitly state. 
Let Zn maintain its previous definition. 

Theorem 7.4. Suppose that 

(7.31) V* = E(Xi \n>i) 

exists for all positive integral i for which P {n > i} 5 ^ 0. In those cases write 

(7.32) p’t = £(1 X, - V.-11 n > 1 ). 

Suppose also that the series 

(7.33) Z(|-; + ••• + = i| 

•-.1 

converges. Then 

“ S"'] “ 

It is regrettable but unavoidable that the mean values Vi and p[ entering into 
(7.33) and (7.34) be conditional. The fundamental reason is that the sequential 
process may drastically modify the distribution of dependent chance variables, 
so that their distribution for our purposes can only be considered in conjunction 
with the sequential process itself. Consider the following example: 

PjX. = -11 - P\Xi = 1) = i 
P\X2 = -2 I X, = -1) = i 

P{X, = -1 I = -1| = i 

P{X2 = 1 I X, = 1| = i 

P|X, = 2 I X, = 1} = }. 

We have E(Xt) = 0. Suppose we define the following sequential process: 
IfX, = —l,n = l.andif Xi = l,n = 2. It is then clear that for our purposes 
Xt can take no negative values and the fact that E{Xi) = 0 is of no use to us. 


(7.34) 


E\ 
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If, however, the chance variables , Zj, • • • are independent, this difiicidtty 
disappears, and we have the following. S 

Corollary 1 to Theorem 7.4. If the chance variables Zi, Z 2 , • * • are mde- 
pendenty we have Theorem 7.4 wiih vi = E(Xi)y and Vi = E \ Z< — I'i | . 

If further all the Z,- have the same distribution, we sec that Theorem 7.1 is 
a special case of Theorem 7.4, since the convergence of the series (7.33) is then 
a consequence of the existence of w and En. From this argument we see, how¬ 
ever, that it is not necessary that all the Z,* have the same distribution, and we 
may write the following generalization of Theorem 7.1: 

Corollary 2 to Theorem 7.4. Let the Z< he independent withy in generaly 
different distributions. SupposSy howevery that all Vi are equaly and all v\ are equally 
except perhaps for those i such that P [n> i] =0. Suppose further that En exists. 
Then (7.1) holds. 

Among possible fields of application of Theorem 7.4 are sequential tests of 
composite statistical hypotheses, and the random walk of a particle governed 
by probability distributions which are functions of time and the position of the 
particle. The extension of this theorem to vector chance variables is straight¬ 
forward. The extension to higher moments may present difficulties. We hope 
to return to some of these questions in the future. 

Proof of Theorem 7.4. This is very elementary. We have 

E(zn - E .i) = z /^^[g (X. - dF(x., • • • , X,) 

(7.35) = E E f (x,- - Vi) dF{xi ,■■■ , Xi). 

y-i Jr{ 


= Ep|« > j\EiX, - 1 n > j) = 0. 

J-1 


The rearrangement of the series is valid because 


(7.36) 


ttf I X; - Vi I dF(x, , • • • , X.) = E>'/-P{»» > 3} 

j-l i-j JRi ;-l 


00 

= Z(n 


+ • • • + v^)P{n. = j\ 


which converges by (7.33). 


REFERENCES 

[1] David Blackwell, “On an equation of Wald,” Annals of Math. Stat.y Vol. 17 (1946), 

pp. 84-87. 

[2] H. CramI&b, Mathematical Methods of Statisticsy Princeton Univ. Press, 1946. 

[3] M. A. Girshick, Frederick Hosteller, and L. J. Savage, “Unbiased estimates for 

certain binomial sampling problems,” Annals of Math. Stat.y Vol. 17 (1946), 
pp. 13-23. 



230 


J. WOLPOWITZ 


[4] H. Cramer, “A contributiuii to the theory of statistical estimation," Skandinavisk 

Aktuarietidakrift, Vol. 29 (1946), pp. 85-94. 

[5] A. Wald, “Sequential tests of statistical hypotheses," Annals of Math, Stal.^ Vol. 16 

(1945), pp. 117-186. 

[6] A. Wald, “On cumulative sums of random variables," Annals of Math, Stat,, Vol. 15 

(1944), pp. 283-296. 

[7] A. Wald, “Differentiation under the expectation sign of the fundamental identity in 

sequential analysis," Annals of Math. Stat,^ Vol. 17 (1946). 

[8] C. H. Uao, “Information and the accuracy attainable in the estimation of statistical 

parameters," Bull. Calcutta Math, Soc., Vol. 37, No. 3 (Sept.. 1945), pp. 81-91. 

[9] A. C. Aitken axd H. Silvehstone “On the estimation of statistical parameters," 

Proc, Hotf. Soc. Edinburgh, Vol. 61 (1941), pp. 56-62. 



ESTIMATION OF LINEAR FUNCTIONS OF CELL PROPORTIONS 

By John H. Smith 
Bureau of Labor Statistics 

Summaiy. In this article certain contributions are made to the theory of 
estimating linear functions of cell proportions in connection with the methods 
of (1) least squares, (2) minimum chi-square, and (3) maximum likelihood. 
Distinctions among these three methods made by previous writers arise out of 
(1) confusion concerning theoretical vs. practical weights, (2) neglect of effects 
of correlation between sampling errors, and (3) disagreement concerning methods 
of minimization. Throughout the paper the equivalence of these three methods 
from a practical point of view has been emphasized in order to facilitate the 
integration and adaptation of existing statistical techniques. To this end: 

1. The method of least squares as derived by Gauss in 1821-23 [6, pp. 224- 
228] in which weights in theory are chosen so as to minimize sampling variances 
is herein called the ideal method of least squares and the theoretical estimates 
are called ideal linear estimates. This approach avoids confusion between 
practical approximations and theoretical exact weights. 

2. The ideal method of least squares is applied to uncorrelated linear func¬ 
tions of correlated sample frequencies to determine the appropriate quantity 
to minimize in order to derive ideal linear estimates in sample-frequency prob¬ 
lems. This approach leads to a sum of scpiares of standardized uncorrelated 
linear functions of sampling errors in which statistics arc to be substituted in 
numerators. 

3. A new elementary method is used to reduce the sum of squares in (2)— 
before substitution of statistics—to Pearson^s expression for chi-square. In 
this result, obtained without approximation, appropriate substitution of sta¬ 
tistics shows that the denominators of chi-square should be treated as constant 
parameters in the differentiation process in order to minimize chi-square in 
conformity with the ideal method of least squares. . 

4. The ideal method of minimum chi-square, derived in (3) as the sample- 
frequency form of the ideal method of least squares, yields ideal linear estimates 
in terms of the unknown parameters in the denominators of chi-square. When 
these parameters are estimated by successive approximations in such a way as 
to be consistent Avith statistics based on them, it is shown that the method of 
minimum chi-square leads to maximum likelihood statistics. 

5. An iterative method which converges to maximum likelihood estimates is 
developed for the case in which observations are cross-classified and first order 
totals are known. In comparison with Deming’s asymptotically efficient 
statistics, it is shown that, in a certain sense, maximum likelihood statistics 
are superior for any given value of n—especially in small samples. 

6. The method of proportional distribution of marginal adjustments is de- 
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veloped. This method yields estimates of expected cell frequencies whose 
efficiency is 100 per cent when universe cell frequencies are proportional—^a 
condition closely approximated in most practical surveys for which first order 
totals are available from complete censuses. Whether this favorable condition 
is satisfied or not, the method yields results which are easy to interpret and it 
has many computational advantages from the point of view of economy of time 
and effort. 

Throughout the article discussion is confined to the estimation of parameters 
whose relationships to cell proportions are linear. However, most of the results 
can be extended to the case of non-linear relationships, the necessary qualifica¬ 
tions being similar to those in curve-fitting problems when the function to be 
fitted is not linear in its parameters. In this case, of course, least squares esti¬ 
mates are not linear estimates. In particular, obvious extensions of the general 
proofs in sections 5 and 6 make them applicable to the non-linear case. Thus 
even when relationships are non-linear, it can be shown that the method of 
minimum chi-square is the sample-frequency form of the method of least squares 
which leads (by means of appropriate successive approximations) to maximum 
likelihood statistics in sample-frequency problems. This principle which 
establishes the equivalence of the methods of least squares, minimum chi-square, 
and maximum likelihood greatly facilitates the integration and adaptation of 
existing techniques developed in connection with these important methods of 
estimation. 

1. Introduction. This article deals with problems of statistical estimation in 
which the parameters to be estimated are cell proportions or linear functions of 
them, A simple illustration of this type of problem is that of estimating p, 
the proportion of white men in a population classified by race and sex. Fom 
a sample of n persons selected at random from such a population, the desired 
proportion can be estimated by simply taking the sample proportion of white 
men as an estimate of the corresponding cell proportion in the population or 
universe. This estimate is unbiased for all possible values of p and its sampling 
variance is p(l — p)/n—assuming, for simplicity, that sampling is done with 
replacements. Whether a more accurate unbiased estimate of p can be derived 
depends on whether or not any other relevant information concerning the cell 
proportions in the universe is available. For example, it may be known that 
all of the white portion of the population is composed of married couples so that 
in the universe the number of white men is exactly equal to the number of white 
women. This knowledge implies that half the proportion of whites provides an 
unbiased estimate of p which is far more accurate than the sample proportion 
of white men. In fact, the sampling variance of half the proportion of whites 
is equal to (2p)(l — 2p)/4n—^less than half the sampling variance of the pro¬ 
portion of white men. 

The term ideal linear estimate will be used to refer to any statistic which satis¬ 
fies the criteria of estimation implied by the foregoing discussion—^that is, an 



ESTIMATION OP LINEAR FUNCTIONS 


233 


ideal linear esimate is any estimate which (1) is a linear function of the sample 
observations; (2) is recognizable as unbiased by the research worker; and (3) 
has minimum sampling variance among estimates which have properties (1) 
and (2). These important criteria of estimation will now be stated in more 
technical language. 

Let ni, 712, and represent the number of (1) white men, (2) white women 
and (3) non-white persons, respectively, in samples of n persons. Since any 
linear function with a constant term can be reduced to the homogeneous form 
by adding an appropriate multiple of the identity 

(1.1) -f- 7^2 "b 7^8 — 7i = 0, 

it is possible, without loss of generality, to confine attention to linear estimates 
of the form 

(1.2) T = ai7li -f 02712 + OsTlj , 

which are recognizable as unbiased. In this example, the research worker is 
assumed to know that the cell proportions in the universe are 

(1.3) Pi, P2, P8 = P, P, 1 - 2p. 

Hence, absence of bias implies that the expected value of T 

(1.4) E(T) = OiTipi 4“ (h'ftpz + ojTips 

~ (oi "b 02 — 2 (in)np "b TiOs 
is identically equal to p; in other words, that 

(1.5) 7i(ai + 02 — 203 ) — 1=0, 
and 


7108 = 0. 

The ideal linear esimate is derived by finding values of 01 , 02 , and Os which 
minimize the sampling variance of T subject to equations (1.5) as side condi¬ 
tions.^ In this way it can be showm that half the sample proportion of whites 
is actually the ideal linear estimate of p. For more general problems, the 
process of minimization of sampling variances with the aid of Lagrange multi¬ 
pliers involves expressions which are complicated algebraically. For this reason 
it is usually easier to derive ideal linear estimates of parameters which are linear 
functions of cell proportions by the ideal method of least squares which is 
presented in section 4. 

Like other least squares estimates, an ideal linear estimate of a linear function 
of cell proportions depends on ideal least squares weights. Since these weights 

^ In this example, it is possible to solve equations (1.5) for as in terms of ai, drop sub¬ 
scripts, and substitute in the formula for the sampling variance of T to obtain a quadratic 
in a to be minimized. 
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are, in general, functions of variances and covariances of sample frequencies, 
the theoretical connotation of the term “ideal'* makes it preferable to other 
terms such as “optimum" and “best." In this connection it should be em¬ 
phasized that (1) the sampling variance of linear estimates is insensitive to 
small errors in estimating ideal weights, and (2) the process of deriving practical 
approximations to ideal linear estimates automatically provides maximum 
likelihood estimates of the ideal weights. Thus the estimation of weights is 
perfectly objective and the best practical approximations to ideal linear esti¬ 
mates are expressed in terms of sample observations. This degree of objec¬ 
tivity is rare in statistical estimation as a brief consideration of regression prob¬ 
lems will illustrate. 

In ordinary regression problems, the ideal weights are inversely proportional 
to error variances. It is usually necessary to draw upon past experience to 
estimate relative weights because satisfactory estimates of error variances 
are rarely available in terms of sample observations. From the present point 
of view, the widespread use of equal weights implies the subjective “assumption" 
that all error variances are equal. (Maximum likelihood estimates of regression 
coefficients require, in addition, the even more subjective assumption of nor¬ 
mality.) In spite of these (usually implicit) subjective assumptions, dis¬ 
cussions of optimum properties of least squares regression coefficients based on 
ideal weights in terms of unknown parameters are highly commendable because 
(1) sampling variance is not very sensitive to small errors in weights and (2) 
properties of theoretical ideal linear estimates furnish a simple basis for dis¬ 
cussion of the properties of practical statistics based on any reasonably good 
approximations to the exact ideal weights. In any case, it is important to 
know what the ideal weights are in terms of unknown parameters because 
research workers can make better estimates if they know what quantities should 
be estimated than they could otherwise. 

2. Estimation of a single parameter. In sample-frequency problems, least 
squares weights ai-e rarely given explicitly or even implied by information 
available to the research worker. Since the hypothetical example used in 
Section 1 is a trivial special case from, this point of view, a more realistic ex¬ 
ample is presented in this section. Since the biological interpretation of this 
problem is presented in detail in all but the first of the many editions of Fisher's 
well-known book [3] it is sufficient here to consider only the statistical problem. 
The four cell proportions are 

(2.1) Pi, P2, Ps, P4 = (2 + e)/4, (1 ~ 0)/4, (1 - <?)/4, e/4, 

and the parameter ^ is to be estimated from the set of sample frequencies 

(2.2) ni, n2, ns, n4 = 1997, 906, 904, 32, 

obtained in a sample of n = 3839 selected at random from an infinite universe. 
Fisher considers five different statistics— Ti , Ts, Ts, T 4 , and Tt —so it will 
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be convenient to use the symbol for the ideal linear estimate. Consider 
the class of linear unbiased estimates of the form 

(2.3) T = cLiTii -j- 4“ CLifli 4” ^4^4 , 

where absence of bias implies that 

(2.4) 2a\ 4“ 4“ ^3 = 0 

and 


rti — <12 — fla 4" ^4 — 4/w = 0. 

Minimizing the sampling variance of T in equation (2.3) subject to side 
conditions based on equations (2.4) yields the ideal linear estimate 7^6 defined 
by the equation 

( 2 . 5 ) nil + 2d)l\ = 3drh - SSn. - 4 - (4 - d)tu. 

The exact sampling variance of 7 e, 


( 2 . 6 ) 


20(1 - 0)(2 4 - e ) 
nil 4- 20) 


is used by Fisher as the asymptotic sampling ^ ariance of any efficient estimate 
of 0. The exact sampling variance of the ideal linear estimate is especially 
appropriate as the asymptotic sampling variance of the maximum likelihood 
estimate Ta because 1 \ is the limit of an iterative process designed to estimate 
'i\ as closely as possible from sample data by using successive approximations 
to Te for 0 in equation (2.5). The limit of this process (wliich is, of course, 
only an approximation to can be obtained by substituting the symbol 74 
for both and 0 in equation (2.5) and .solving the resulting quadratic equation 
which can be reduced to 


(2.7) nT\ — (/ii — 2?i2 — 2n3 — u^Ta — 2n4 = 0, 

an equation which is identical, except for notation, with Fisher’s equation of 
maximum likelihood of which Ta is the positive solution. 

The foregoing result is a comparatively simple illustration of the general 
principle that the maximum likelihood estimate of any linear function of cell 
proportions is the limit of an iterative process designed to approximate the 
corresponding linear estimate as closely as possible by means of sample fre¬ 
quencies. Since the accuracy of estimates of least squares relative weights 
increases with size of sample, maximum likelihood statistics have, in an asymp¬ 
totic sense for large samples, the same optimum properties which are possessed 
in an exact sense (even for small samples) by the corresponding ideal linear 
estimates. Thus the results obtained by means of the theory of large samples 
are supported by the approach to estimation problems by means of ideal linear 
estimates. In addition, the later approach facilitates the integration of 
available techniques as explained in later sections. 
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It is true that the optimum properties of maximum likelihood statistics can 
be presented in terms of the theory of large samples, but the fact that a given 
method of estimation yields a statistic whose asymptotic sampling variance is 
a minimum does not imply that the same technique will yield a minimum 
variance statistic for any given small value of n. For example, it is well known 
that the median is a maximum likelihood estimate of the midpoint of a double 
exponential universe. Nevertheless, in samples of three observations from 
such a universe, another statistic—4/9 of the mean plus 5/9 of the median— 
has greater relative advantage over the median than the median has over the 
mean. 

Fisher^s discussion of the relative efficiencies of his five alternative consistent 
statistics suggests that it is impossible to formulate objective criteria for making 
choices among alternative statistics such that each statistic will be used whenever 
its sampling variance is smallest. Consider the sequence of universes generated 
by letting 6 vary from zero to unity. In general, each value of 6 would deter¬ 
mine which of Fisher’s five statistics would have smallest sampling variance 
for that particular universe for any given value of a. In comparison with 
any other single statistic, the statistic Ta would usually have smaller sampling 
variance, but there are notable exceptions. For example, in the absence of 
linkage when 9 is equal to one-fourth, the statistic T 2 is the ideal linear estimate 
and its sampling variance is smaller than that of Ta —at least for certain small 
values of n. For this reason, Fisher used T 2 in preference to 7^4 as the basis for 
testing the significance of linkage. The statistic Tb —derived by Fisher’s method 
of minimum chi-square—^is also of special interest. Fisher’s method of minimum 
chi-square yields statistics which differ from the corresponding maximum 
likelihood statistics because Fisher considers the denominators as variables in 
the process of differentiation instead of considering them as unknown para¬ 
meters to be estimated by identif 3 dng them with the corresponding statistics 
in the numerators after differentiation. Arguments of later sections tend to 
show that the latter method is more appropriate. In this example, it can be 
shown that if were substituted for the corresponding parameter in the de¬ 
nominators of chi-square (and treated as a 'parameter) the minimization of chi- 
square with respect to statistics in its numerators would be exactly equivalent 
to substituting 0.035785, the numerical value of for 6 in equation (2.5) and 
solving for to obtain 0.035717, a value which is much closer to 0.035712, 
the numerical value of the maximum likelihood estimate Ta than to Fisher’s . 
In problems of estimation chi-square should be minimized in order to obtain 
eflScient statistics—not to obtain a small criterion for testing goodness of fit— 
and it should be minimized in a manner consistent with this purpose. Whether 
or not it is possible to derive an even smaller value for a quantity called chi- 
square should be considered to be irrelevant in either estimation problems or 
tests of significance. It is difficult to present these ideas in more technical 
language because it is possible to construct trivial hypothetical universes for 
which Fisher’s method of minimum chi-square provides statistics which are 
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superior in certain respects to the corresponding maximum likelihood statistics. 
Nevertheless, it seems clear that the ideal linear estimate usually has smaller 
sampling variance than the maximum likelihood statistic which, in turn, usually 
has smaller sampling variance than any other given practical statistic. Evi¬ 
dence presented in later sections tends to show that these advantages are more 
important in small samples than in cases in which the theory of large samples 
is applicable. 

3. The “ideal” method of least squares. When sample observations are 
uncorrelated in successive samples and parameters to be estimated are linear 
functions of the expected values of the sample observations, the method of least 
squares yields ideal linear estimates of the parametes provided that the weight of 
each observation is inversely proportional to its variance in successive samples. 
Although the minimum sampling variance property among linear unbiased 
estimates is seldom stressed, this principle of weighting has been presented in 
connection with the method of least squares for more than a hundred years. 
In order to emphasize the theoretical nature of weights which depend on vari¬ 
ances which are usually unknown in practice and to distinguish the method 
based on such weights from the more familiar method of least squares with 
equal weights, the method which yields ideal linear estimates will be called the 
ideal method of least squares. 

Discussion of the general problem of estimating linear functions of cell pro¬ 
portions can be facilitated by making use of results obtained by other writers— 
notably Gauss (as reported by Whittaker and Robinson [6]) and Pearson [4], 
According to Whittaker and Robinson, “the first writer to connect the method 
[of ideal least squares] with the theory of probability was Gauss” [6, p. 224]. 
In his Theoria Motus proof of 1809, Gauss derived the “most probable value” 
[6, p. 223] of a parameter (i.e., the statistic which satisfies the criterion now 
called maximum likelihood) for the case in which sample observations are sta¬ 
tistically independent and normally distributed. In his Theoria Combinationis 
proof of 1821-23, Gauss “abandoned the ‘metaphysicar basis” [6, p. 220] of 
his earlier work and derived the method herein called the ideal method of least 
squares (without approximation) from the criteria of (1) minimum variance and 
(2) absence of bias for the case in which “the mean value of [the covariance of 
a pair of errors] is zero” [6, p. 224]. Since the covariances of uncorrelated linear 
functions are zero whether they are statistically independent or not, it follows 
from the work of Gauss that the ideal method of least squares applied to un¬ 
correlated linear functions of sample frequencies yields ideal linear estimates. 
In other words, the ideal method of least squares implies the following six steps: 

1. From the set of A; -h 1 sample frequencies construct k linear functions 
which are uncorrelated in successive samples. 

2. From each function subtract its expected value in terms of the unknown 
parameters to find its sampling error. 
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3. Write the ratio of each sampling error to its own standard error in the' 
form of a fraction. 

4. Sum the squares of these standardized uncorrelated sampling errors to 
obtain a quantity called chi-square. 

5. Substitute statistics* for the parameters in the numerators of chi-square. 

6. Minimize the sum of squares of residuals with respect to each statistic 
in turn (subject to appropriate side conditions in case linear functions 
not implied in preceding steps are known). 

This series of six steps can be summarized by the single statement that the 
function to minimize is the sum of squares of standardized uncorrelated resid¬ 
uals. Actually this statement is oversimplified because even though sampling 
errors are both uncorrelated and standardized» the corresponding residuals 
are, in general, neither standardized nor uncorrelated. 

4. Pearson’s expression for chi-square. As defined by Pearson [4], chi- 
square is the sum of squares of a set of standardized uncorrelated linear func¬ 
tions of sampling errors in a set of fc + 1 correlated sample frequencies. A set 
of k standardized uncorrelated linear functions can be constructed in an infinite 
number of \\ ays, but each set can be obtained from any of the others by means 
of an orthogonal transformation. Thus the sum of squares is the same no 
matter what set is originally chosen. As his set of standardized uncorrelated 
linear functions, Pearson chose those determined by the axes of the correlation 
ellipse for which he gave the required sum of squares in terms of “minors” or 
cofactors of the correlation determinant of the first k sample frequencies. Pear¬ 
son reduced this complicated expression to the now familiar form 

(4.1) X* = 22 («. - npif/np,, 

»—1 

where is the proportion in the ith cell in the universe and n. is the frequency 
in the fth cell of a sample of n observations selected at random from an infinite 
universe (or with replacements from a finite universe). 

The widespread misunderstanding of the nature of chi-square seems to be 
based primarily on the facts that 

1. Pearson’s rule for degrees of freedom is inadequate (sec section 5), and 

2. Pearson’s expression for chi-square can be derived by approximate methods 
as well as by exact methods. 

Pearson’s derivation of the expression for chi-square by exact methods is suf¬ 
ficient to show that its derivation by approximate methods involves a paradox 
in which different sets of approximations offset each other; however, Pearson’s 
article is relatively inaccessible and, in addition, his algabraic reductions involve 

* It is convenient to call these variable symbols statistics”; the quantities whose 
squares are summed, “residuals”; and the whole expression “chi-square,” even though, 
from a certain point of view, these terms are strictly applicable only after the minimiza¬ 
tion process. This usage should always be clear from its context. 
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the minors of a general determinant of the kth order. For these reasons, the 
following exact derivation is presented in terms of elementary algebra. 

Since the sum of squares is the same for any set of k standardized uncorrelated 
linear functions of the sampling errors in + 1 correlated frequencies, a set should 
be chosen for which the algebraic reductions are as easy as possible. From this 
point of view a satisfactory set, which can be written in any of three forms, is 
given by 

(4.2) y, = - p,+n. 


= - (p. + p,+)e, 

where c, = n» — npt and i+ and refer to classes formed by combining all 
classes above the zth class and below the tth class, respectively. 

By means of the known variances and covariances of the sample frequencies 
in expected value form, 

(4.3) E(e\) = nptii - p^), 
and 

(4.4) Eie,e,) = -np,pj, 


it can be shown that the variance of y, is 

(4.5) E{y,) = np.p,+(p, + Pt+), 

and, by using the third expression in equation (4.2) for y* and the second for 
y,, it can be shown that any pair of y^s are uncorrelated because 

(4.6) E{y,y;) = 0, {i < ;). 


Let Zt represent the variable y, expressed in standard-deviation units. The 
square of this standardized uncorrelated linear function of correlated sampling 
errors can be written 


(4.7) 


^2 _ (p» 

* ” Wp»p,+(p. + Pt+) ‘ 


It remains to show that Pearson's expression for chi-square can be obtained 
by adding the k values of z\ in succession. For this purpose it is convenient 
to define 


(4.8) 


2 V 

Xr = 2^ — + 

t.i np% 


ir±- 

npr+' 


obtained by combining all classes above the rth class. 

When r = k, the expression in equation (4.8) is the expression to be derived. 
It remains to show that xl is the sum of squares of k standardized uncorrelated 
linear functions of sampling errors; i.e., 
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(4.9) Xfc = 2? • 

For the first cell ei+ = — and = 1 — pi. Hence yi reduces to the negative 
of the error in the first frequency and 

(4.10) Xi = €?/npi(l - pi) 

= ei/wpi + el+lnpi^ (pi+ = 1 - pi), 

a special case expressed in the required form. The general case is established 
by showing that 

(4.11) xU + zl = xl , 


or, alternatively, that 

-2 2 2 
Zr = Xr — Xr-1 

= el/npr + eU/npr+ - (Cr + er+Y/n{pr + Pr+) 

(4 12) = Pr ^r-{-)(Pr 4~ Pr-f) VrPr-^j^r "t" 2gr ^r-f- gr+) 

^PrPr+(Pr + Pr+) 

= Pr4.4. — 2prPr+erer+ + Pr-H^r = (Pr " Pr4-gr)^ 
nprPr+(Pr + Pr-t-) ' 


thus establishing the derivation of Pearson^s expression for chi-square. 

When sampling is done without replacement each variance and covariance 
is multiplied by {N — n)/{N — 1) where N is the number of observations in 
the universe. Hence, chi-square for this case can be written 


(4.13) 


2 


X 


N - 1 ^ ^ 
N — npt ’ 


This expression shows that the factor involving sampling errors is the same 
whether sampling is done with replacement or without replacement. Hence, 
the derivation of least squares statistics is the same for either method of sampling, 
but sampling variances for the simpler case are multiplied by the factor {N —• n)/ 
(N — 1) when sampling is done without replacement. 


6. The method of minimum chi-square. The derivation of Pearson’s ex¬ 
pression for chi-square completes first four steps of the ideal method of least 
squares outlined in section 3. Hence, the method of minimum chi-square is 
the sample-frequency form of the ideal method of least squares in which only 
two of the six steps refnain to be taken. 

In his original article [4] Pearson pointed out that the use of statistics instead 
of parameters would affect the value of chi-square but that such effects would 
usually be so small that no allowance need be made for them in connection with 
tests of significance. It is now well known that the average value of chi-square 
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is reduced approximately one unit for each parameter estimated from the sample, 
and that the main portion of this effect is on the numerators; i.e., in large samples 
the effect of substituting statistics for parameters in the denominators usually 
has a negligible effect on the value of chi-square. By confining the discussion 
to the case in which parameters are used in the denominators, it is possible to 
make simple exact statements concerning the main effects in terms of the number 
of squares of standardized uncorrelated linear functions—also known as the 
number of degrees of freedom and the mean value of chi-square. 

When the expected values in the numerators of chi-square can be expressed 
as linear functions of r algebraically independent parameters, ideal linear esti¬ 
mates of the r parameters are determined by substituting statistics for the r 
parameters and minimizing the resulting expression wth respect to each sta¬ 
tistic. In general, such a substitution of statistics for parameters in the numer¬ 
ators of chi-square reduces the number of degrees of freedom by one unit for 
every parameter estimated; that is, the appropriately minimized chi-square 
can be analyzed into ~ r squares of standardized uncorrelated linear functions 
of sampling errors. 

The r ideal linear estimates are linear functions of the sample frequencies. 
T.»et (I’l, , • • •, ^f) be a set of standardized uncorrelated linear functions of 

the correlated sampling errors in these statistics and let (vi, V 2 , * • •, vi) be a set 
of linear functions obtained from the z,^s of section 3 by an orthogonal trans¬ 
formation, Since the sum of squares is not changed by such a transformation, 
chi-square is the sum of the k values of v\ . The process of substituting statis¬ 
tics for the r parameters in the numerators of chi-square reduces the values of 
the first to zero without affecting the values of the other {k — r)t;<*s. 

Thus the appropriately minimized chi-square can be analyzed into k -- r 
squares of standardized uncorrelated linear functions of sampling errors and is 
therefore said to have k — r degrees of freedom. The mean value of each square 
is the variance of a standardized linear function of sampling errors and is there¬ 
fore unity by definition. Hence the mean value of the appropriately minimized 
chi-square (with parameters in the denominators) is exactly k — r when r 
statistics are estimated from a set of + 1 sample frequencies. 

The expression to be minimized is 


(5.1) 


(fij - m'i)^ 
npi 


where m[ is the ideal linear estimate of np,. The set of statistics described 
by the equation 

(5.2) ) 


reduces the value of chi-square to zero—^its minimum value. This shows that 
the sample cell proportion is the ideal linear estimate of the corresponding 
parameter. 

Whenever a linear function independent of the sum of the cell proportions is 
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known, it is possible to take advantage of additional information provided by 
the known function by minimizing chi-square subject to an appropriate side 
condition. When side conditions are used in this way, the number of degrees 
of freedom for the minimized chi-square is equal to the number of side conditions 
which are algebraically independent of each other (and of the sum of the cell 
proportions). Let the known linear function be written 

(5.3) SOinpi — m = 0. 

In order to facilitate comparison of the typical equation of maximization 
with the corresponding equation of the method of maximum likelihood, it is 
convenient to minimize chi-square by maximizing — x^/2 subject to a side 
condition based on (5.3). The function to be maximized can be written 

(5.4) — xV2 = 2(ni — m^)V(“2np.) + h{Za{mi — m), 

where h is a Lagrange multiplier. Setting the partial derivative of — x^/2 
with respect to equal to zero, the typical equation for minimizing chi-square 
can be written 

(^.5) {iii “ 7ni)/npi -f htti = 0, 

a form which shows that, in general, ideal linear estimates are defined in terms 
of unknown parameters. Fortunately, these parameters can usually be approxi¬ 
mated closely by an iterative process. Substituting rrii for both np< and 
in equations (5.5) the typical equation in the limiting values of such a process 
can be reduced to 

(5.G) 7ii/nii — 1 + htti = 0, 

a form which is identical with the typical equation (6.6) of maximum likelihood 
derived in section 6. This equality of typical equations implies that whenever 
the denominators of chi-square are estimated in such a way as to be consistent 
with least squares statistics based on them, the method of minimum chi-square 
always leads (by means of approximations necessary in practice) to maximum 
likelihood estimates of parameters which are linear functions of cell proportions. 

6. The method of maximum likelihood. Maximum likelihood estimates of 
linear functions of cell proportions can be obtained by (1) expressing the prob¬ 
ability function (general term of the multinomial expansion) in terms of the r 
parameters to be estimated; (2) substituting r statistics for the r parameters; 
and (3) maximizing with respect to the r statistics. In practice, this is usually 
accomplished by maximizing the logarithm of the variable factor in step (3) 
which can be written, 

(6.1) L = 2n<logmi, 

where rrii is the maximum likelihood estimate of np,-, the expected value of the 
ith frequency n,* in a sample of n observations classified into {k -f 1) classes or 



ESTIMATION OF LINEAR FUNCTIONS 


243 


cells. It is evident that L as written has no maximum with respect to any m, 
since it increases without bound as wi,increases, but it sometimes has a uniquely 
determined maximum when each of the m^’s is written explicitly in terms of 
less than A; + 1 algebraically independent statistics. In the general case it is 
easier to maximize L subject to an appropriate set of side conditions, one of 
which must be equivalent to 

(6.2) wii + m 2 + • • • + mjt+i — n = 0. 

When no linear function except the sum is known, the likelihood function 
can be written 

(6.3) L = Sn, logm, — (Sm, — n), 

a function which, subject to equation (6.2), is always equal to that in equation 
(6.1) but which has a uniquely determined maximum. The t 3 rpical equation of 
maximum likelihood, obtained by setting the partial derivative of L with respect 
to m* equal to zero, is 

(6.4) n,/m, — 1=0, 

an equation which shows that each sample frequency is a maximum likelihood 
estimate of its own expected value. 

When a linear function such as that in equation (5.3) is known, an improved 
set of maximum likelihood statistics can be found by maximizing 

(6.5) L = 2n,logm, — (2m, — n) + h{Zam% — m). 

The typical equation of maximization is found to be 

(6.6) n,/m, - 1 + ha, = 0, 

an equation which, as stated above, is identical with equation (5.5). Since 
equation (5.5) was obtained as the limit of an iterative process from the typical 
equation (5.4) for minimizing chi-square subject to the same side condition 
and since each additional side condition affects the typical equation of each 
method in exactly the same way, the method of minimum chi-square and the 
method of maximum likelihood are equivalent for the general case in the sense 
that the method of minimum chi-square always leads to maximum likelihood 
statistics as limits of an iterative process. 

7. Second-order tables with known expected marginal totals. As stated in 
section 2, the integration of available techniques is facilitated by regarding 
maximum likelihood statistics as the best practical approximations to the 
corresponding ideal linear estimates. Since this important principle may not 
be immediately obvious, it will be illustrated for the important special case of 
second-order tables for which the expected marginal totals are known. 

Consider a sample of n observations arranged on two bases of classification 
and presented in a table containing r rows and s columns. The universe of N 
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observations has been completely enumerated and classified on each basis 
separately but not cross-classified; i.e., universe totals of first order classes are 
known. 

For the cell in the ith row and the ^‘th column, let p.y represent the universe 
cell proportion; the sample frequency; np<j, the expected value of 
and rriijy the maximum likelihood estimate of np,y. Indicating summation 
by substituting a dot for the letter over which summation is to be performed, 
the known marginal totals satisfy the equations 

(7.1) Npi, - Ni. = 0, 

Npj - N„ = 0 , 

where p*-. and p.j are the universe proportions and Ni. and N,j are the known 
universe totals in the tth row and the ^*th column, respectively. 

When n observations of a random sample are arranged according to two 
bases of classification in a table with r rows and s columns for which the r -f « 
marginal totals are known, the typical equation of maximum likelihood can 
be obtained by maximizing, subject to side conditions based on equations (7.1), 
the likelihood function 

(7.2) L = 2Sn*jlogm<y — Xaiinii, — Ui.) — — n.;), 

with respect to the maximum likelihood estimates w,-/, where a< and 5, are typical 
Lagrange multipliers. Setting the partial derivative with respect to equal 
to zero and transposing, the typical equation of maximum likelihood can be 
written 

(7.3) ~ bj. 

Since equations (7.3) are not linear in their unknowns, the reader^s first 
reaction might well be to agree with a certain anonymous critic that “their 
solution is difficult.” This impression of great difficulty is probably the chief 
reason that previous writers have not used the method of maximum likelihood 
for this type of problem even after they had developed a set of techniques ade¬ 
quate for the solution of the equations of maximum likelihood. In other words, 
all that was needed was the integration of available techniques as will now 
be shown. 

In 1940, Doming and Stephan [2J derived a set of normal equations for the 
adjustment of a set of second-order cell frequencies to known expected marginal 
totals by the method of least squares in which each sample frequency is weighted 
by its own reciprocal. This method yields statistics which are efficient according 
to the theory of large samples, but they do not satisfy the criterion of maximum 
likelihood exactly. In the same article was presented an easier method of 
iterative proportionsy which, unfortunately, does not yield least squares sta¬ 
tistics. In 1942, Stephan [5] developed an improved iterative process which 
yields statistics which satisfy the criterion of least squares with arbitrarily 
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chosen weights. The foregoing developments are presented in greater detail 
in Deming^s book [1] in which Deming adapts Stephan’s iterative method to 
the particular case in which each sample frequency is weighted by its own 
reciprocal so as to yield solutions for the normal equations derived in the joint 
article [2], 

In Deming’s notation, equation 8 of Stephan’s article [5, p. 169] can be written 
(7.4) rriij = Cii(p< + g,- — 1) + n,y, 

an expression obtained by substituting c</ for npa in the denominators of chi- 
square and minimizing with respect to the statistics in the numerators. Hence, 
if exact values of the np.y were used for the Cij , the Stephan iterative method 
would yield ideal linear estimates. Unless these parameters are implied by 
some hypothesis to be tested, it is necessary, in practice, to estimate the np<y 
from sample data. In order to secure maximum likelihood estimates of expected 
cell frequencies by means of the Stephan iterative method, the adjusted fre¬ 
quencies based on first approximations to the c,y should be used as second ap¬ 
proximations to the Ctj , etc. In this way, maximum likelihood statistics can 
be derived to any desired degree of approximation. At this point it should 
be emphasized that the preceding statement applies not only to the class of 
problems considered in this section but also to the wider class of problems for 
which the Stephan iterative method provides solutions. 

Unfortunately, theoretical discussions of previous writers contain confusing 
compensating errors which (I) present their own methods in an unnecessarily 
unfavorable light and (2) increase the difficulties involved in the introduction 
of the improvements in techniques suggested in section 9 which involve some 
degree of adaptation of techniques already available. For these reasons, it 
seems necessary to follow the arguments of previous writers in order to show 
the points at which improvements are needed. This can be done most effec¬ 
tively in connection with Deming’s book [1] where the method of least squares 
is presented in great detail. 

For the special case in which the sampling errors in the observations are un¬ 
correlated, the ideal criterion of least squares implies that the weight of each 
observation should be inversely proportional to its sampling variance. This 
criterion is accepted as well known by Deming who says that “the principle of 
least squares requires the minimizing of the sum of the w^eighted squares of the 
residuals” [I, p. 14] where “the weights of two functions are inversely pro¬ 
portional to their variances” [1, p. 22]. Deming assumes that “there is no 
correlation between the errors in the observations” with the qualification that 
“this assumption covers a wide class of problems, but does fail to cover some.” 
[1, p. 49]. This assumption of uncorrelated errors is not applicable to sample- 
frequency problems, of course, because the sample frequencies are correlated 
\vith each other in such a way that the reciprocals of the ideal least squares 
weights are not proportional to the sampling variances npijQij but rather to 
the expected frequencies np.y whict appear in the denominators of chi-square. 
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In this connection it is interesting to note that Deming himself insists that 
“there is only one principle of least squares, namely, the minimizing of x*-” 
[1, p. 51]. However, the method currently in use for the minimizing of chi-square 
was that given by Fisher [3] which leads to equations which are difficult to solve 
even for such a simple example as the one presented in section 2 above. 

Deming and Stephan are to be commended for seeking an easier method 
but there is no justification (even as a device for saving effort) for their modifica¬ 
tion of the “principle of least squares” so as to imply erroneously that 

(1) weights of correlated sample frequencies are inversely proportional to 
their variances, and 

(2) sample frequencies are, in general, approximately proportional to their 
own sampling variances. 

Strangely enough, these two errors were applied in combination by Deming and 
Stephan to obtain good practical approximations to the ideal least squares 
weights. It might be argued that the second misleading implication is really 
not an error because it is offered as a simplifying approximation, but it is an 
integral part of both the normal equations approach in the joint article [2] 
and Deming’s adaptation [1] of the Stephan iterative method; that is, in each 
case the method would have to be revised if better approximations to the ideal 
least squares weights were used. More explicitly, Deming (1) uses n*, for Ste¬ 
phan’s Cij in equation (7.4); (2) identifies it with the other n,-, in the same equa¬ 
tion; and (3) reduces the equation to a different form thus effectively preventing 
the use of successive approximations to the c.y without returning to Stephan’s 
iterative method in the general form given by equation (7.4) above which 
Deming does not present at all. Results of the joint article [2] are quoted by 
Stephan [5] without any explanation of the nature of the errors, but none of 
these results are used in the development of his iterative method which as noted 
above, is applicable to any arbitrarily chosen set of weights. The fact that 
Stephan corrected the second error without correcting the first implies that the 
weights he actually used are unsatisfactory. In Deming’s adaptation of the 
Stephan iterative method, a much better set of weights is obtained, not by cor¬ 
recting the first offsetting error overlooked by Stephan, but by resurrecting the 
second offsetting error which Stephan had corrected. Since this error is an 
integral part of Deming’s adaptation, Deming’s theoretical discussion implies 
that his own efficient statistics are only rough approximations which are definitely 
inferior to the inefficient statistics obtained by means of the weights chosen by 
Stephan. These inconsistencies are most clearly brought out by Deming when 
he says: 

“Strictly, in random sampling, the reciprocal of the weight of n*, is npaqa , which is 
nearly equal to nt,9<y where p and q have their usual connotations. But since factors pro¬ 
portional to the weights may be substituted for them, it is sufficient to use na as the re¬ 
ciprocal of the weight in cell tj, since the values of qa do not usually vary much over the 
table.“ U, p. 102.1 

In any given problem the seriousness of the error in the first statement in 
the foregoing quotation depends on the variation among the ^ty’s. In the par- 
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ticular example used by Deming the error is of considerable importance because 
the largest is more than 40 per cent larger than the smallest qij . The weights 
actually used by Deming agree with weights implied by the ideal method of least 
squares except for sampling errors in the w.-y; hence, the error in any relative 
weight converges stochastically to zero so that Deming^s statistics are efficient 
according to the theory of large samples. The efficiency of Deming’s statistics 
is inconsistent with the theory presented by Deming which implies erroneously 
that efficiency of estimation depends on approximate equality of cell proportions. 
If this argument were true it would apply also to the method of maximum 
likelihood and all other methods which yield efficient practical statistics in 
sample-frequency problems. The foregoing discussion, together with the results 
of section 8 show that the theory as presented by Deming has the following 
seriously misleading features: 

(1) it is based on a paradox in which a good final result is obtained by means 
of compensating errors; 

(2) it presents his efficient statistics in an unnecessarily unfavorable light; 

(3) it emphasizes the irrelevant condition of approximate equality of universe 
cell proportions; 

(4) it fails to mention the important condition of proportionality by rows 
and columns; and 

(5) it makes least squares, minimum chi-square, and maximum likelihood 
seem to be competing alternative methods. 

Of these undesirable characteristics, the last two are probably the most serious 
because they make the effective integration and adaptation of statistical tech¬ 
niques more difficult. As has been shown in sections 4, 5, and 6, the sample- 
frequency form of the ideal method of least squares is the method of minimum 
chi-square which always leads (by means of appropriate practical approxima¬ 
tions to unknown weights) to maximum likelihood statistics; in other words, 
the methods are equivalent from a practical point of view. 

Since the ideal method of least squares based on the unknown determines 
fully efficient, but theoretical, ideal linear estimates, the efficiency of practical 
approximations to ideal linear estimates depends on the accuracy with which 
the denominators of chi-square are estimated. For the unknown denominators 
npij , Deming uses the sample frequencies n,j while the method of maximum 
likelihood implies the use of the corresponding maximum likelihood estimates— 
statistics which, in general, have smaller sampling variances. The foregoing 
argument suggests that maximum likelihood statistics are slightly superior to 
Deming’s statistics for any given finite value of n and that their relative ad¬ 
vantage increases as the sample size decreases. In large samples both methods 
yield efficient statistics because the relative errors in the weights implied by 
either method converge stochastically to zero as n increases. Although the ad¬ 
vantage of maximum likelihood statistics over Deming’s statistics is unim¬ 
portant except in small samples, it can be shown that Deming’s choice of weights 
leads to imperfectly compensated negative errors of estimation even in his 
large sample of 33,837 observations. 
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Deming weights each sample frequency by its own reciprocal. Positive errors 
of sampling decrease the value of the reciprocal and thus increase the absolute 
size of the required negative adjustments. Negative errors of sampling increase 
the value of the reciprocal and thus decrease the size of the positive adjustment. 
Thus every error of sampling (either positive or negative) leads to a negative 
error of estimation due to inappropriate weighting. Because the sum of all 
adjustments must be zero, these negative errors of estimation are compensated 
on the average but more or less imperfectly. The net effect of this imperfect 
compensation of negative errors of estimation is that Deming’s statistics are 
too small in those cells in which the relative adjustments (either positive or 
negative) are large, and vice versa. In a preliminary draft of this article, 
this type of error of estimation was studied by comparing Deming’s statistics 
Avith the corresponding maximum likelihood statistics in conection with Deming\s 
example involving 33,837 observations. Although errors of estimation of the 
type under discussion are apparent, they are, of course, extremely small in such 
a large sample. For this reason the large-sample comparson has been deleted 
in favor of simple hypothetical examples designed to throw light on similar errors 
of estimation in statistics derived by Fisher^s method of minimum chi-square 
as well as in those derived by Deming’s adaptation of Stephan’s iterative 
method. 

Consider a set of sample frequencies in a two-by-two table for vhich all 
expected marginal totals are equal. For this special case, the cell proportions 
on each diagonal are equal and the ideal linear estimate (which is also the 
maximum likelihood estimate) of any cell proportion is the mean of the two 
sample cell proportions on its diagonal. For the same case, Deming’s adaptation 
of the Stephan iterative method yields an estimate for each cell which is pro¬ 
portional to the harmonic mean of sample proportions on its diagonal while 
Fisher’s method of minimum chi-square yields estimates proportional to the 
corresponding quadratic means. 

As a numerical example of the foregoing problem consider the set of fre¬ 
quencies 

(7.5) nil, ni2, n2i , n22 = 1,4, 3, 2, 

obtained in a sample of 10 observations selected at random from a univei*se 
in which the cell poportions are known to be 

(7.6) pii, pi2, P21, P22 = p, 0.5 - p, 0.5 - p, p. 

As estimates of the parameter p, the ideal linear estimate is .15, Deming’s 
adaptation of the Stephan iterative method yields .14, and Fisher’s method of 
minimum chi-square yields .1545 to four decimal places, the other two estimates 
being exact. The results illustrate the imperfectly compensated errors of 
estimation explained previously. The two sample frequencies on the principal 
diagonal {nu and n 22 ) have greater relative dispersion than the frequencies on 
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the other diagonal. For this reason, the relative adjustments made by Deming’s 
method are greater and according to the principle of imperfectly compensated 
negative errors of estimation, the estimate of p obtained by Deming’s method 
is smaller than the ideal linear estimate of p. Fisher’s method of minimum 
chi-square yields an estimate of p which is greater than the ideal linear estimate. 
In fact, one should usually expect imperfectly compensated errors of estimation 
in statistics derived by Fisher’s method of minimum chi-square to be opposite in 
sign and about half as large as those in the corresponding statistics derived by 
means of Deming’s adaptation of the Stephan iterative method. 

At this point, it should be emphasized that Fisher does not recommend his 
own method of minimum chi-square in preference to the method of maximum 
likelihood. In fact, he presents the theory of estimation in such a way as to 
imply correctly that the method of maximum likelihood is superior, especially 
in small samples. Other writers have noted the small differences between 
equations of maximum likelihood and those for minimizing chi-square by Fisher’s 
method and some have even derived one set of equations from the other by 
neglecting higher order terms in a Taylor series expansion. These derivations 
are of no interest here because they seem to justify the method of maximum 
likelihood as a simple approximation to some more complicated method. This 
type of justification is both unnecessary and undesirable. It is more useful to 
regard the method of maximum likelihood as an approximation to a method— 
least squares—for which the theory is simpler. 

Skeptical readers who find the foregoing argument unconvincing may be able 
to profit from the following example. Consider the problem of estimating the 
parameter p where 2p is the proportion of white balls in an urn. A sample of 10 
balls is selected and classified by the following process. Each white ball is 
placed in one of the cells on the principal diagonal of a two-by-two table, the 
particular cell being decided by the toss of a coin. A similar method is used for 
non-white balls placed in cells on the other diagonal. Assuming that the results 
of this process arc given by equation (7.5), which of the three alternative esti¬ 
mates of p given above should be preferred? Belief in the general superiority 
of Fisher’s method of minimum chi-square seems to imply that the device of 
coin-tossing described in this example can be used in practical problems involving 
the estimation of the proportion of ‘‘successes” to secure estimates which are 
superior to the sample proportion—^the ideal linear estimate in such cases. 
Even if it is possible to construct trivial special case examples supporting some 
complicated method for such problems the general use in practical problems of 
the coin-tossing device in connection with either Fisher’s method of minimum 
chi-square or Deming’s adaptation of the Stephan iterative method would be 
absurd as this example is intended to emphasize. 

8. The method of proportional distribution of marginal adjustments. The 

method of proportional distribution of marginal adjustments is a general method 
of adjusting sample frequencies so that their row and column totals agree with 
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known expected marginal totals. In other words, the adjusted frequency for 
the cell in the tth row and the yth column is given by the equation 


(8.1) 

»»<# = «<)• — Pi.p.i — P.i di., 

where 

di. = mi. — fii., 

and 

d.i = m.j - n.j. 


are the net adjustments in the sample cell frequencies of the fth row and the 
yth column, respectively. The asterisk is used to distinguish maximum likeli¬ 
hood estimates mu and the ideal linear estimates m<y from the set of statistics 
based on equation (8.1). 

The method of proportional distribution of marginal adjustments yields ideal 
linear estimates when the universe cell proportions are proportional by rows and 
by columns; i.e., when 

(8-2) Pa = Pi.p.i. 

This important principle can be established by substituting in equation (7.4) 
of section 7 the quantities 

(8.3) Co = npi.p.y. 

Pi = 0.5 + di./npi ., 

and 


gy = 0.5 + djnp.i, 

and reducing the typical equation of the ideal method of minimum chi-square 
to the form of equation (8.1) which defines the method of proportional dis¬ 
tribution of marginal adjustments. 

Even in the absence of exact proportionality, under which it yields fully 
efficient statistics, the method of proportional distribution of marginal adjust¬ 
ments has the following relative advantages over other available methods: 

(1) ease of extension to tables of higher order; 

(2) exact agreement with known (expected) marginal totals; 

(3) simplicity of interpretation; 

(4) independence of computational errors; 

(5) rapidity of processing; 

(6) economy of effort; and 

(7) fully efficient criteria for testing the significance of departures from 
proportionality of rows and columns. 

Ease of extension to tables of higher order is a desirable property of the 
method of proportional distribution of marginal adjustments. Equation (8.1) 
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applies to the special case in which there are only two bases of classification. 
In the more general case sample observations are cross-classified according to 
r bases of classification, each cell frequency in an rth order table being the num¬ 
ber of observations in the corresponding rth order class whose expected value 
is to be estimated. The required adjustment for each first order class (obtained 
by subtracting the sample total from its known expected value) is distributed 
among the various cells in proportion to the universe totals of the corresponding 
(r “ l)th order classes to which the cells belong. The general process is il¬ 
lustrated by 

(8.4) ntijk = ^ijk + Pi..d.jk + p.j.dt.k + p,.kdij ., 

the formula for estimating the expected frequency in the general cell of a third 
order table. 

Exact agreement with marginal totals follows easily from the method of 
proportional distribution and can be established algebraically by summing the 
estimation equation by first order classes; e.g., summing equation (8.1) byrows 
and columns. In practice, discrepancies are always either errors of rounding 
or mistakes in computation ; they are never due to lack of convergence of iterative 
processes as is often true in alternative methods of estimation. 

Although simplicity of interpretation is desirable in general, it is especially 
important when random sampling is an unrealistic abstraction. For example, 
the method of proportional distribution of marginal adjustments has been used 
to estimate the cell proportions in a two-way classification of incomes from known 
marginal proportions and a detailed cross classification at an earlier date. In 
this problem known shifts in income distributions made it evident that certain 
cells previously vacant should not have the zero proportions which would be 
estimated for them by other available methods of estimation. The ease with 
which the effects of the method of adjustment can be traced is important also 
in the analysis of the results of sample surveys in which various types of bias 
are important. 

The method of proportional distribution of marginal adjustments yields the 
estimated expected frequency for any cell by a single sequence of computations 
which is independent of the corresponding process for any other cell. Errors 
made in computing the estimate for any cell appear in marginal totals of esti¬ 
mates for all first order classes which include that cell. If only a few errors are 
made in a table they can be localized immediately and can be corrected without 
recomputing any estimates which are correct. 

In certain t 3 rpes of social surveys, rapidity of processing is so important that, 
as Deming puts it, ‘‘the delay of only the brief time required for adjustment 
may not be advisable.” [1, p- 102]. Under these conditions, it is important to 
have a simple formula like equation (8.1) in which substitutions can be made 
rapidly. Even when the time element is relatively unimportant, the economy 
of effort and the ease of explaining the method to clerical assistants are often 
of practical importance. 
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Finally, departures from proportionality among rows and columns often 
provide the chief element of interest in research studies—^not only in social 
surve 3 rs of the type illustrated in Deming and Stephanas example but also in 
biological sciences. The most elffective tests of significance for the purpose of 
presenting statistical evidence of lack of proportionality are those based on 
statistics like those derived by the method of proportional distribution of marginal 
adjustments whose efl&ciency is 100 per cent when proportionality is exact. 

Even when proportionality is not exact, the efficiency of statistics derived 
by proportional distribution may be close to 100 per cent under fairly typical 
problem conditions such as those in the example by Deming and Stephan wherein 
the other more complicated methods require several times as much computational 
effort, but have little advantage over the easier method mth respect to effi¬ 
ciency of estimation in this particular problem. 

9. Suggested improvements in techniques. In section 7, a method was 
outlined by which it is possible to derive sets of maximum likelihood statistics 
by merely integrating available techniques without changing any of them. 
In this section a number of improvements are suggested. At this point it should 
be emphasized that a given change is not an improvement merely because it 
yields slightly more accurate estimates or makes possible a slight saving of 
time and effort. In each case the research worker should consider saving of time 
and effort and accuracy of estimation simultaneously. In particular, it seems 
likely that most social surveys of the type considered by Deming and Stephan 
are characterized by approximate proportionality by rows and by columns— 
conditions relatively favorable to the simple method of proportional distribu¬ 
tion of marginal adjustments. It should be clearly understood that sug¬ 
gestions in this section are intended for those research workers whose problems 
justify a great deal more effort than is required to adjust sample frequencies 
by this simple method. 

Assuming that the problem at hand warrants the effort required to derive 
maximum likelihood estimates, the first consideration is the derivation of a 
set of first approximations to the m<y, and a set of values of pi(l), 

first approximations to the . Even if proportionality by rows and by columns 
is not closely approximated use of values of thep*(l) provided by equation (8.3) 
are especially to be recommended. In the example used by Deming these 
values for the pt(l) are so much better than the values recommended by Deming 
that they save a large proportion of the effort required by the iterative process. 
If rows and columns are approximately proportional, equation (8.1) should be 
used to provide values of the ?Wti(l), in which case it is possible to use an itera¬ 
tive process similar to the one used by Deming but based on the typical equa¬ 
tion of maximum likelihood (7.3) to achieve a given degree of accuracy in the 
maximum likelihood estimates with even less effort. Under favorable conditions 
such as those in Deming’s example the suggested iterative process yields excellent 
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approximations to maximum likelihood estimates by means of the following 
steps: 

1. Construct a set of first approximations to the r row components of the rs 
maximum likelihood divisors (o» + 6y) by means of the equation 

(9.1) 0,(1) = Uiju'pi, - 1/2. 

2. Compute successive approximations to the at and bj by means of the equa¬ 
tions 

(9.2) hjig) = [n,j - Sm,i(l)a.(gr)]/wp.y, 

(9.3) ai(g + 1) = [n*. - 2m.y(l)6y(gr)]/np»., 

where miy(l), the first approximation to m^y, is derived by means of equation 
(8.1). Just as in Deming’s iterative process, the expression in brackets is a 
series of products which can be subtracted in a single sequence of machine 
operations and the final division can be performed without having to record 
any of the intermediate results. 

3. Divide the sample frequencies by the maximum likelihood divisors to obtain 
the maximum likelihood estimates 

(9.4) m,y = riij/iai -f 6y), 

where limiting values of a, and bj are approximated as closely as desired by 
successive approximations in the preceding equations. 

Under unfavorable conditions, the iterative process of this section is not 
always the easiest way to obtain satisfactory estimates. For example, when 
samples are small and/or rows and columns are not approximately proportional, 
it is better to use the iterative method as originally presented by Stephan where 
sample frequencies can be used for first approximations to the Cty and these may 
be replaced by successively better approximations. 

The point made in the final paragraph of Fisher’s well-known book [3] that 
“in practice one need seldom do more than solve, at least to a good approxima¬ 
tion, the equation of maximum likelihood,” is strongly supported by the develop¬ 
ments of this article. In addition, the proof that the method of least squares 
and the method of minimum chi-square always lead (by means of approxima¬ 
tions to ideal weights) to maximum likelihood statistics greatly facilitates the 
adaptation of techniques developed in connection with these hitherto competing 
methods. 
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A STATISTICAL PROBLEM CONNECTED WITH THE COUNTING OF 
RADIOACTIVE PARTICLES 

By Sten Malmquist 

Institute of Statistics, University of Upsala, Sweden 

1. Introduction. Our problem refers to random events forming a sequence 
in time or in space, e.g. particles emitted by a radioactive matter. By omitting 
certain elements of the given sequence, say /, we form another sequence, say g. 
The rule of omission involves an arbitrarily prescribed constant u. The rule 
to be followed in forming g is: 

Case I: Let a be an element in / and g. The next element to be included 
in ^ is then the first element in / which follows a after a distance greater than u. 

Case II: Let a be an element in / an&^gf. The next element to be included in 
g is then the first element in / which follows a at a distance greater than u from 
the preceding element in /, whether this belongs to g or not. 

AVhen the events are represented by impulses emitted by a radioactive matter 
and feeding a recorder with a constant resolving time u, the new sequence con¬ 
sists of the counted impulses. The two cases correspond to the reaction of 
different types of recorders. The distinction between the two transformations 
has caused some confusion. It has, however, been clearly pointed out by 
Ruark and Brammer [5]. 

V. Bortkiewicz [2] seems to be the first who has considered problems related 
to the transformed sequence. Starting from investigations by Rutherford, 
Geiger, and others, concerning the number of recorded a-particles during a 
certain interval of time, say T, he observed that the distribution of this number 
was similar to that of Poisson but with a slightly smaller dispersion. This fact 
he supposed to be caused by a constant resolving time u of the recorder. By 
means of certain assumptions he tried to calculate the effect on the mean and 
the dispersion by the transformation in Case 1, supposing the cumulative dis¬ 
tribution function F{t) for the distance between two consecutive elements in 
the sequence/is given by 

Fit) = 1 - 

where here and in what follows, t denotes a non-negative variable. 

Considering Case II with Fit) as above, Levert and Scheen [4] have recently 
worked out an expression for the distribution of the number of elements during 
T in the sequence g, 

Gnedenko [3] has considered the distribution of the number of lost elements 
in Case I with particular regard to the initial state of rest. 

Alaoglu and Smith [1] considered problems referring to successive trans¬ 
formations of a sequence. When, for example, a sequence of particles enters 
a tube-counter and amplifier, together acting with a resolving time ui , and 
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the impulsea then are feeding a recorder with resolving time > Ui, tlie se¬ 
quence of recorded impulses will be the result of two successive transformations. 
If we have a scaling circuit between the counter and the recorder, we have to 
make a transformation of another type between the two transformations in 
Case I and Case II. 

The present paper deals with the transformed sequence in Case I. The 
distribution function F{t) is supposed to be arbitrary. An advantage of this 
generalization is that the formulas derived could be used in treating problems 
referring to successive transformations. 

The author wishes to express his sincere gratitude to Professor Herman Wold 
for stimulating discussions and valuable advice. 


2. Derivation of distributions for case L Suppose that the sequence / 
has F(0 for distribution function for the distance between two consecutive 
elements. F(t) is supposed to be independent of absolute time (space), and of 
the preceding distance between two elements. When not stated otherwise, 
we further suppose F(0) = 0. 

Now let G{t) be the distribution function for the distance between two con¬ 
secutive elements in the transformed sequence g. Evidently G{t) also is inde¬ 
pendent of absolute time and of the preceding distance between two elements. 

We shall consider certain distribution functions connected with F{t). These 
functions will then be used in solving problems concerning the sequence g. 

Let F«(0 be the distribution function for the distance between the first and 
the last of n -f 1 consecutive elements in the sequence /. Then Fn(0 is given 
by the recursive system 


( 1 ) 


= [ F„(t — x) dFn(x); 
JQ 

Fid) ^ F{t). 


As is easily seen, we have 


(m, n ^ 1) 


and, for t = u, 

F»{u) —» 0, as n —» 00 ; 

00 

^Fn{u) < 00 , provided that Fi(0) <1. 

nawl 

Alternatively, F„(0 could be deduced by the use of characteristic functions. 
Still considering the sequence /, let be the distribution function for the 
distance d between an arbitrarily chosen point and the following element. 
Suppose that the arbitrary point is chosen so that the distance between the pre- 
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ceding and the following element is x. Under this condition we have, in usual 
S3m:ibols, 


P(d > 0 = 


Hence, 


m = ^-1 


where H(t) is the distribution function for the distance x. 

To deduce JI(i) we suppose that the distribution F(t) has a finite mean, 


= f tdF(t). 


By the definition of H{t), we then have 


H(x) = - r tdFil). 
m Jo 




The corresponding frequency function <pit) is given by 

.<0 . 

Consider n + 2 consecutive elements in /, say Oo, > * * * , On+i, where Oo 

is an element in the transformed sequence g. The probability Pn that the 
next element in g following oo will be an+i is given by 


Pn = Fn{u) - Fn+i(u), 

Po = 1 - F{u). 


(n = 1, 2, • • • ), 


Now let Pn{t) be the probability that the distance between oo and Gn+i is 
smaller than or equal to t, when Oo an Un+i are two consecutive elements in the 
sequence g. Then 

- F.i„) 


(n = 1, 2 • • •). 


o _ m - Fiu) 
" -1 - F(«) • 


Let G*{t) be defined by 

Q*(t) ^'EPn- PniO = FCO - F(v,) 

nmmO 


+ 5 i‘ [F(« -X) - F(m - *)] dF»(x); t > «. 



258 


STEN MALMQXnST 


When G*{t) is a distribution function, then G*{t) equals G{t), 

For ti < ti we obviously have G*(ti) < 

For / = 00 t 

Gf*(oo) = 1 - F(u) + E ffl - F(u - x)]dFn(x) 

n-1 Jq 

= 1 - F(u) + - EFn+lW = 1. 

1 1 

Hence we take 

(4) Git) = G*it); t > w, 

Gii) =0; ^ < u» 

When the corresponding frequency functions git) and Sit) exist, we get 

(6) gif) = fit) + 'll i fit- x)fnix) dx; t > u. 

n—1 Jo 

Dealing with a sequence of elements we are often concerned with the number 
of occurrences during a certain time T. 

Let the mean number of occurrences during T be MiT). Supposing that 

the mean ^ t dFit) is finite and that F(0) < 1, we have 

(6) MiT) = T/m. 

We define 

Kiit) = 

K,it) = 

and denote the corresponding means by 

Miit) < Mit) < Mj(e). 

Uang (2), 

tFjt) + e[l - F(e)] _ 


Fit) 

for t 

> 

■— e 

0 

for t 

< 6 

Fit) 

for t 

> € 

Fit) 

for i 

< c 

MiiT) and MiiT). 

As is easily seen, 


Miit) = 


[ x dKiix) X dKiix) ' 


Miit) = 


X dKiix) 


- Fit)]^ + • •. + n-*[l - Fit)]^Fit)’'-^ + • • 


X dKiix) 
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Making N = T/e and summing, we obtain 


My{T) = 


f X dKi{x) m — f X dF(x) + eFie) 
Jq Jo 


MiiT) 


_ T_ _ 

f xdKiix) 

Jo 


m — 


T 

f X dF(x) 
Jo 


By choosing e arbitrarily small, we get 

M(T) T/m. 

Let P(n, T) be the probability that we get n elements in / during a time T. 
Suppose that the first of these elements, ai , comes at To + x, and the last, 
an , at 2o + ic + y- 
We then have 

(7) P(n, T) = <p{x) dx [I - F(T -X- y)] dF^-iiy). 


In (4) and (7) we have equations for the transformation in Case I. Because 
of the general form of Fit)j the formulas also can be used when we are concerned 
with successive transformations. It can further be remarked that the trans¬ 
formation of a sequence of impulses by passing a scaling circuit is expressed by 
the system (1). 


3. Results for a particular form for F (/). The preceding formulas will 
now be used for a special distribution function F{t). Suppose that the fre¬ 
quency function fit) = dFit)/dt is equal to the frequency function of the dis¬ 
tance between an arbitrary point and the following element. 

From (3) we get 

1 - Fit) 


nt) = 


m 


or, when F(0) = 0, 

( 8 ) 


(9) 


Fit) = 1 - 

fit) = where 1/a = rw = tfit) 


dt. 


By means of the theory of characteristic functions we have 

/i(0 ss/(0; 


( 10 ) 

where 

( 11 ) 


“ 2ir L 


nix) = a / 
Jo 


—at itx 


dt 


a — vt 
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1 


(o — m:)” 


Thus 

( 12 ) 

Forn = 1, we get 
(13) 


dx 


dx 


By differentiating (13) n — 1 times with respect to a we obtain 


(-<)' 

Hence, from (12), 
(14) 


n—1 '-at 

e 




(o — tx)" 


dx. 


m = 




(n- 1)!' 

From (6) we obtain the frequency function for the transformed sequence g 


(16) 


ff«) = ae-“‘ + i: f 

fi-l 


ae 


(n - 1)1 
G(f) = 0; t <u. 


t”~^dx 


t > u 


The mean trig is given by 

r 

nig ^ a I 
Ju 


e dt —\r u. 


Remark: Suppose the constant u is allowed to vary independently of t and 
that the frequency function of u is 7 (w), we obtain 


(16) 


mg — I tdtf g(Uy t)y(u) du = f - y(u) du + f uy(u) du 
Jo Jo Jo d Jo 


= - + m{u), 
a 


Now let the sequence of elements, g, by means of (5) be transformed into a 
new sequence, h. When we are concerned with the counting of particles, 
emitted from a radioactive matter, let the sequence g consist of impulses from 
a counter-amplifier with resolving time w, feeding a recorder with resolving 
time Ui . Then the elements in h are the counted impulses, it being supposed 
that the tube-counter and the recorder reacts according to the assumptions. 

We suppose Ui > u. When Ui < Uy the sequences g and h are identical. 

Let gn{t) denote the frequency function of the distance between the first and 
the last of n + 1 consecutive elements in g. We find, in the same way as 
used in obtaining (14), 


(17) 


Unit) = 


(n - 1)1 


e‘"“(« - n«)"-^- 


,n—1 j—ai 


t > nu. 
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Let h{t) be the frequency function for the distance between two consecutive 
elements in the sequence h. Let further N be the greatest integer smaller 
than or equal to ui/u. 

Using (4) and (5) we obtain 


A,(0 = 'E-Aui- 

0 n! 

(18) had) = ^ - (n + 

0 n\ 

hm{i) = E ^ [^ - (» + 1^6°’"', 

0 711 


t ^ ui + u; 
{N + l)u < t < ui + u\ 
< t < (N l)w. 


The mean is found to be 

(19) m. = ri + a] r 1 + i: i: e-"'”*-"”’ 1. 


We also have 


or 


f Ihi(l) dt < rtih < f tki(l) 

Jui + u •'U 

-] 


dt 


a(ui~nu) 


< W). < [*- + «il e°“ Ce —, («i — nu) 
La J L 0 


n —o(ui--»u) 

V 


]■ 


We now consider the number of occurrences during a time interval T, Using 
(6), (16), and (19) we immediately get the mean numbers of occurrences during T, 
By (3), we get for the sequence g 


( 20 ) 




OW + 1 ’ 


t < u 


CL au —at . 
C 6 • 


t > U, 


QU -j- 1 

Inserting (20), (15) and (14) in (7) and evaluating the integrals, we finally get 


(21) P,(n, T) = \ 


0»-i — 20n + On+1 ; 

Gn-l — 2an + (n + 1) — 
aT 


aT 


o„_i - 2 j^n 


au + 


au + I' 


T 

n ~ 1 
u 

-1 < n < ~ 

u u 


T T 

± < n < i- + 1. 

u u 
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where 

( 22 ) 


On = 


-a(r-nu) — nuY 


au + I 
0-1 = 0 . 


E 

vhO 


ti! 


(to - »), 


(to = 0, 1, • • •), 


When w = 0, we obtain 

n rpv V 

0« = c"*’’ ^ (to - v). 

V.0 Vl 

For the sequence / we then get the Poisson distribution 

(23) Pf(n, T) = e""'’. 

n\ 

The corresponding expression for the sequence h is much more complicated* 


4. A statistical experiment. The following statistical experiment will serve 
as an illustration of the scheme dealt with in this paper—the transformation of 
a sequence and the resulting formulas, especially (21). 

Groups of five figures, the last rounded up if necessary, have been extracted 
from tables of random sampling numbers (6). Let each group denote the first 
five digits for a decimal x, arbitrarily chosen between 0 and 1. The variable 
X is supposed to have the distribution function Hor 0 < ^ < 1. We now define 
a new variable, y, given by 

(24) y = -A;log(l - j), [or y = -klogx]. 

The variable y has the distribution function given by (8), viz. 

F{t) = 1 — where - = m = fcloge. 
a 

Transforming each group, or number x, according to (24), we get a sample of 
consecutive distances between elements in the sequence / considered in the 
previous sections. Choosing a constant w, we can construct the corresponding 
sequence g. Beginning with a point, arbitrarily chosen on the first distance, 
we can finally count the number of elements in successive intervals of the same 
length. 

Take k = 1, w = 0.2 and T = 1.5. We then have for the sequences/ and g: 

m/ = - = log e = 0.4343; rrig — - + u = 0.6343; 

a a 

C/ = -=> 0.4343; (r, = i = 0.4343; 

a 

M,{T) = ^ = 2.365. 

TUg 


M,{T) = L = 3.464. 
rrif 
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The experiment yielded the following results: 

For the sequence f: For the sequence g: 

Number of elements 801. Number of elements 565. 

M/ = 0.450. fhg = 0.648. 

In neither case is the deviation between the observed and theoretical means 
statistically significant. In fact we have: 

(in/ - m,)V^ _ ^ Q, (m, - w,)-y/^ _ ^ g 

<r/ ‘ ’ Cg 

which gives P = 0.3 and P = 0.4, respectively. 


TABLE I 

Nos. of intervals with n elements 



Sequence/ 

Sequence g 

71 

Observed 

Expected 
according 
to (23) 

Observed 

Expected 
according 
to (21) 

Expected 
according 
to (23) 

0 

6 

7.0 

5 

8.2 

23.7 

1 

33 

26.1 

53 

42.5 

54.8 

2 

48 

45.1 

82 

81.8 

63.3 

3 

55 

51.9 

69 

72.2 

48.8 

4 

36 

44.8 

23 

29.21 

28.1 

5 

32 

31.0 

6 

4.8[ 

13.0 

0 

17 

17.8 

1 

O. 2 J 

5.0\ 

7- 

12 

14.7 



2 . 4 / 

S 

239 

239 

239 

238.9 

239 

Mean 

3.331 

3.454 

2.310 

2.36 

2.31 



4.825 

j 


4.524 

36.7 

P 


0.68 


0.34 

<0.001 


The functions On in (22) can be calculated by means of Pearson’s tables of 
the incomplete y-function (7). In the notation of these tables we obtain 


Hence 


au +1 nl au l 
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where 

\ = aiT - nu); p = ; g = n - 2. 

In the present case, however, we only need the numbers up to ay. Accordingly, 
the a„ have been calculated directly. 

The resulting theoretical and observed distributions for the number of ele¬ 
ments during T for the sequences / and g will be found in Table 1. For com¬ 
parison, a Poisson distribution, with the same mean as ol)served for the se(iuence 
g\ is given. The result of a test is also shown in Table I. Judged by the x 
test the distributions (23) and (21) agree fairly well with the observed distri¬ 
butions. As was to be expected, the Poisson distribution cannot be used for 
the sequence g. 
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THE PROBABILITY FUNCTION OF THE PRODUCT OF TWO NORMALLY 
DISTRIBUTED VARIABLES' 

By Leo A. Aroian 
Hunter College 

1. Introduction and summary. I^ot x and y follow a normal bivariate prob¬ 
ability function with means X, Y, standard deviations ay , a-i , respectively, r 
the coefficient of correlation, and pi = X/ay, = F/<r 2 . Professor C. C. 
Craig [1] has found the probability function of z = xyjayfji in closed form as 
the difference of two integrals. For purposes of numerical computation he has 
expanded this result in an infinite series involving powers of 2 , pi , p 2 , and Bessel 
functions of a certain type; in addition, he has determined the moments, semin- 
variants, and the m()ment generating function of 2 . However for pi and p 2 
large, as Craig points out, the series expansion converges very slowly. Even 
for Pi and p 2 as small as 2, the expansion is unwieldy. We shall show that as 
Pi and P2 —> , the probability function of 2 approaches a normal curve and in 

case r = 0 the Type Til function and the Gram-Charlier Type A series are excel¬ 
lent approximations to the 2 distribution in the proper region. Numerical in¬ 
tegration provides a substitute for the infinite series w^herever the exact values of 
the probability function of 2 are needed. Some extensions of the main theorem 
are given in section 5 and a practical problem involving the probability function 
of 2 is solved. 


2. Theorems on approach to normality. The moment generating function 
of 2 , is [1] 


( 2 . 1 ) 


(pi + P2 — 2rpyp2)B^ 4 - 2piP20 

M (R) = 2[l-(l+r)g][l + (1 -r)gf 

\/[l - (1 + r)9][l + (1 - r)e] 


Let 2 , and o-, be the mean and the standard deviation of 2 , and L = (z — 2)/<r,. 
Now 


(2.2) z == P 1 P 2 4“ — V^pf 4" P 2 “f" 2rpip2 4- 1 4- 

Using (2.2) we find in the usual w^ay the moment generating function of L 


(2.3) M,. = 


exp 


—2rw 4" (pi 4- P 2 + 2rpip2)w^ 4- 4rV 2w^(r^ — l)(piP2 4- r ) 
2[1 — (1 4 - r)tf;][l 4 - (1 — r)w] 


y/[\ — (1 4- r)u;]tl 4- (1 — r)w] 


where w = B/<rz • 


^ Presented to the American Mathematical Society, Oct. 28, 1944, New York City. 
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Consider r ^ 0. Then in the limit as pi and p 2 in any manner whatever, 
(2.4) Um M,Se) = e"'*, 

and by the theorem of Curtiss [2] on moment generating functions we see in 
the limit as pi, pa —> <» the probability function of z approaches a normal curve 
^vith mean, I, and variance , r ^ 0. 

In case - l-fe <r<0, €>0, some eai-e is I'equired wherever 

Vpi + pI + Spipar 

occurs. If one uses pi + Pa ^ 2pip2, the proof goes forward quite readily. 
Hence we have proved the theoi*em: 

Theorem (2.5). The distribution of z approaches normality with mean z, 
and variance crj as pi and pa —> in any manner whatever ^ —l + c<r^l, 
€ > 0 . 

It is evident in Theorem (2.5) we may allow pi, pa —> — without any other 
changes. Theorems (2.6) and (2.7) are proved in essentially the same way 
as (2.5). 

Theorem (2.6). The distribution of z approaches normality with mean 2 , 
and variance <rJ,t/pi-->oo,p 2 —^— 00 ,—1 ^ r < 1 — e, c>0. 

Theorem (2.7). The distribution of z approaches normality if pi remains 
constant pa — 1 + € < r ^ \y t > 0; or if pi remains constant pa —> — », 

-1 ^ r < 1 - 6, € > 0. 

Naturally in any of the theorems pi and pa may be interchanged. In practice 
Pi and p 2 are usually positive. The approach to normality is more rapid if 
both pi and pa have the same sign as r. 


3. Numerical values. In order to show how closely the Type III and the 
Gram-Charlier Type A series approximate the probability function of r, f{z), 
or more precisely/(«, pi, pa, r), we use numerical integration where 


(3.1) 


/(a, Pi, P 2 , r) = /j(«) - h{z), 

^ i " 20^) - P*) 


+ 


a-«■)}!■ 


and 72 ( 2 ) is the integral of the same function over (“ <», 0), [1]. Now Ii(z) 
may be written as 


(3.2) 

where 


<p(t) 


* vr- 7 » f ^, 


V2^ ’ 


X — Pi 

Pitz) = e**, ^3 


rtih. 
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We readily obtain I\(z) \/l — r* by forming the product of 
and 1 /x using numerical integration applying Weddle’s formula, the Gregory- 
Newton formula, or the simple rectangular formula depending on circumstances. 
The rectangular formula [3] is remarkably accurate when the function T = 
in the interval 0 to qo or 0 to — oo is somewhat symmetrical. 
Appropriate tables for (see [41), (see [5]) and \/x (see [ 6 ]) are 

readily available. In the important case of the independence of x and i/, r = 0 
and (3.2) becomes 

(3.3) /,(*) = r , h = x- P,, = 

•'0 X X 


4. Approximations to f(z). When r = 0, the standard seminvariants fo, 
and f 4 of z are 


(4.1) 

remembering 


. _ 6pip2 

^ ipi+pi + ir’ 


t = ^l^(pi + P 2 ) + 1} 

(pi + + D* 


z = P1P2, = 's/p? + p^ + 1 . 

In the Pearson system (see [7]) 6 , the criterion, is 


(4.2) 


2^4 3^8 

6 + {4 


and for the probability function of z 

('4 3) s = 2(pi + P2 + l)l2(pi + P2) + 1| — 18p?p^ 

(pi + P2 + l)[(pi H“ p2 + 1)* + 2(p? “h pi) + 1] 

and if Pi = p 2 = p 

, __ 2(4p^+ l)(2p*-M) - 18p" 


(4.4) 


(2p* + l)[(2p* + D* + (4p*+ !)]• 


Now 6 = 0, fs 7 ^ 0, for the Type III function, and clearly lim 5 = 0. 

By use of (3.3) the accurate values of f(z) have been calculated for various com¬ 
binations of Pi and p 2 and compared with the Type III approximation using 2 , 
fa. 

(4.5) Investigations so far completed show that for pi ^ 4 and p 2 ^ 4 simul¬ 
taneously, and I 5 I ^ .008, the Type III approximation ^vill provide values 
of tg correct to three significant figures at least where 

-ip) ^00 

(4.6) / f(tg) = a, / f(tg) = a, and .05 ^ a ^ .005. 

These are the values of tg which would be needed in testing hypotheses. The 
exact values of t^g^ and for for various values of pi and p 2 less than 4 will be 
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determined it is hoped in the future and will be published along with the com¬ 
parisons of the Type III values of tz with the accurate values of tz in the im¬ 
portant borderline cases of pi = p 2 = 2, and pi = P 2 = 3. The values of }{z) 
for Pi = P2 = 2 and pi = p2 = 4 have been calculated but these are being with¬ 
held for a more complete table. The table of values of z, <Tz y f 3 , { 4 , and 5 
(Table II) ohows then that the Type III function is excellent along a band about 
Pi = P2 , since {3 ^ 0 , and 8 is very small. 

Wc use the Gram-Charli('r Type A series of three terms to approximate the 
probability function of z in /* units. 

(4.7) /(/.) ~ v(t) - If /'*«) + If 

in the usual notation. 


TABI.E I 


tg 

f(ta) Correct value 

Normal Curve 

Gram-Charlier 
Type A 

.9950372 

.2406367 

.2431716 

.2408235 

1.4925558 

.1275209 

.130970 

.127484 

1.9900744 

.0538243 

.0550708 

.053704 

2.4875930 

.0184006 

.0180791 

.0184500 

2.9851116 

0052477 

.0040338 

.0052944 

3.4820302 

.0012009 

.0009272 

.0012804 

3.9801488 

.0002011 

.0001449 

.000260 

4.4770074 

.0000407 

.0000177 

.0000425 

4.9751800 

.00000745 

.00000108 

.00000555 


(4.8) For 1 ^ 3 1 < .5 and {4 < .4 simultaneously the Gram-Charlier Type A 
series is quite adequate for finding probability levels such as those of (4.6). 
These will in general give 3 significant figures for or In the special case 
Pi = 0, p 2 == 10, the Gram-Charlier Type A series differs from /(O very slightly 
in the range 1 ^ | | (see Table 1 ). Naturally the Gram-Charlier will 

be used wherever Type III is not indicated, although there exist some over¬ 
lapping regions where either one may be used. It should be noticed that the 
approach of f{z) to normality is more rapid along a row than down a diagonal. 
In case either pi or p 2 is negative, we may make use of the equation 

(4.9) /(«, -Pi, P 2 , r) = /(-«, Pi, P 2 , -r). 

We note that when r = 0, f{z, pi, P 2 ) always possesses a discontinuity at 2 = 0, 
(see [ 1 ]). A table of z, <r,, f 3 , f 4 , and 8 is provided for values of pi and p 2 from 
0 to 10 inclusive. 
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TABLE II* 


\ 

\ 

pa 

2 

4 

6 

8 

10 


0 

0 

0 

0 

0 


2.236068 

4.123106 

6.082762 

8.062258 

10.049876 

0 

0 

0 

0 

0 

0 


2.160 

.685121 

.319942 

.183195 

.118224 


.529 

.205 

.101 

.059 

.039 


4 

8 

12. 

16. 

20 . 


3 

4.582576 

6.403124 

8.306624 

10.246951 

2 

.8 

.498784 

.274256 

.167493 

.111531 


1.259259 

.557823 

.289114 

.172653 

.113742 


.020 

.056 

.056 

.042 

.031 



16. 

24. 

32. 

40 



5.744563 

7.280110 

9. 

10.816654 

4 


.506408 

.373206 

.263374 

.189641 



.358127 

.224279 

.147234 

.102126 



-.0084 

.0049 

.014 

.016 




36. 

48. 

60 




8.544004 

10.049876 

11.704700 

G 



.346314 

.28373 

.224503 




.163258 

.118224 

.087272 




- .0054 

-.00083 

.0038 





64. 

80 





11.357817 

12.845233 

8 




.262088 

.226472 





.092663 

.072507 





-.0034 

-.0015 






100. 






14.177447 

10 





.210551 

.059553 






-.0023 


* The first value in a cell is 2, the second <73, the third {3, the fourth {4, the 
fifth «. 
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5. Some extensions. We may generalize our results to any case where x 
and y are distributed approximately in a normal distribution such as the dis¬ 
tribution of the product of two means, when the sizes of the samples Ni and N2 
are large and consequently pi and p2 will be large. Another example occurs if 
X and y each follows a Bernoulloi probability function with parameters pi and 
P2 respectively where the number of trials in each case is large. We must warn 
the reader that the condition pi «>, p2 alone does not mean that the dis¬ 
tribution of z approaches normality. Both x and y must be distributed normally. 

The actual problem which gave rise to this investigation was the question 
of determining the sum of a great many variates [8]. Let T variates Vi , 1^2, 

• • •, Vr be given whose sum A = is desired. Clearly 

A = 3 Tp , Fp = E Vi/T. 

i^l 

Now let us estimate A by ^ = faVa where Ta is an estimate of T and V, is an 
estimate of Vp . Tf is very small, pi = will be large and p2 = Vploy^ 

= y/NVpjap will be very large. Assuming Ta is distributed normally and 
obviously Va is distributed normally for N large, we see by the theorems of this 
paper that A will be distributed normally. Confidence limits for A may be 
calculated in the usual fashion as i[ i y^x, where 7 is determined by 

f = a, 

with a generally chosen as .025 or less and 

a2 = -\- V'l, a^ + (Xp (Xf . 

Stratification is also possible. It is interesting to note that many functions which 
occur in life insurance are products. Such applications will be treated fully 
elsewhere. Naturally the critical region whether both tails or one tail of the 
distribution should be used depends on the alternatives to the hypothesis being 
tested. 

Generalizations of the main theorem are possible for the probability function 
oi z — IlJ-i where xi, 0:2, • • •, Xr follow a multivariate normal probability 
function. These will be investigated in a later paper. It may be noted that 
J. B. S. Haldane has investigated the distribution of a product along different 
lines [ 9 ]. 
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NOTES 

This section is devoted to brief research and expository articles on methodology 
and other short items. 


A REMARK ON CHARACTERISTIC FUNCTIONS 

By a. Zygmund 
University of Pennsylvania 

1. Let F{x)y —00 <a;<+oo,bea distribution function, and 

+00 


/ i-00 

e*'* dF{x) 

00 


its characteristic function. It is well known that the existence of ^'( 0 ) ^oes 
not imply the existence of the absolute moment 

( 1 ) f^\x\dF{x), 

J—00 

A simple example is provided by the function 

where C is a positive constant. Since the series on the right differentiated term 
by term converges uniformly (see [1]), ^'(0 exists (and is continuous) for all 
values of t, and in particular at the point ^ = 0 . Obviously (p(t) is the char¬ 
acteristic function of the masses C/2n^\ogn concentrated at the points ±n 
for n = 2, 3 , • • •. The constant C is such that the sum of all the masses is 1. 
The divergence of the series Ul/nlogn implies that in this particular case the 
moment (1) is infinite. 

In a recent paper (see [ 2 ], esp. p. 120 , footnote), Fortet raises the problem of 
whether the existence of <p'( 0 ) implies the existence of the first algebraic moment 

( 2 ) f xdF{x) = lim [ xdF{x), 

•t-oo X-^+oo •/—X 


The main purpose of this note is to show that this is so. We shall even prove 
a slightly more general result. 

A function ^(/) defined in the neighborhood of a point k is said to be smooth 
at this point if 

lim + h) + \pito — h) - 2^(A)) _ ^ 

fc-*+o h 

Clearly, if ^ has a one-sided derivative at the point , the derivative on the 
other side also exists and has the same value. Thus the graph of \l/(t) has no 
angular point for ^ = <o, and this explains the terminology. If ^'(/o) exists and 
is finite, \l/(t) is smooth for i . The converse is obviously false, since any 
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function whose graph is symmetric with respect to < = <o is smooth at that 
point. 

Theorem 1. If the characteristic function ip{t) is smooth at the point 0, then 
a necessary and sufficient condition for the existence of ^'(0) is the existence of the 
moment (2). The value of (2) is —up' (0). 

In particular, the existence and finiteness of v?'( 0 ) implies the existence of ( 2 ). 
That the converse is false, is obvious. For if oo , ai, 02, * * • are positive num¬ 
bers and ao + 2ai -f- 2(12 + • • * = 1, then ^(f) = Oo + 22?* a^ cos nt is the 
characteristic function of the distribution function F{x) corresponding to masses 
concentrated at the integer points ±n and having the values a^ there. Owing 
to the symmetry of the masses, the number (2) exists, and is zero even if 
is non-diffcrentiable for f = 0 (we may e.g. take foir (p{f) the Weierstrass non- 
diffcrentiable function C Zf a” cos 6”^, where C is a suitable constant). ^ 

Proof. We may write 

v ?(0 = f cos xt dG(x) + i f sin xt dG(x) = \l/i(t) + itl/2(t) 

JO •'0 

where 
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Since ^i(^) is even, the smoothness of ^(f), and so also of at the point 


1 = 

0 implies that ^((0) exists and is zero. If /i —> +0, 


^ (h) — ^(0) 
h 

/‘“sin xh \ 

aIIH ^flO 

= +/ =Ah + Bh, 

Jo Jl/h 



/ /.2/A /.4/A 


\B, 

1 < rW \dH\ < hr 

'(/ '^^ + / 

dG + / dO "t" • • * ) 


Ji/h 

\Jl/h J2/h 

• Ji/h ) 



= h 

-‘o(/i +V2 +V4 + ---) = 0(1), 

by 1 

(3) and (5). Also 




f^iih 

Ah - xdH = 

Jo 

/&in hx . 

J» \ hx 

)xdH=j 0{x^h*)xdG 

i 


f''* 



= 

/ 0(x^h.) dG = 

Jo 

o(l), 

by (3) and (4). Thus 




h W — ^ 2 ( 0 ) 

.»/» fllh 

= o(l) + / xdH = 0(1) + / xdF, 


h 

Jo 

J-’Vh 

and 

so 





- o(i)+ i rr dF. 

n J-i/h 


It follows that the existence of (2) is equivalent to the existence of the right- 
hand side derivative of <p{t) at the point t = 0, or, on account of smoothness, 
to the existence of ^'(0). Moreover, the value of (2) is — This com¬ 

pletes the proof of Theorem 1. 

2 . Suppose that a function ^(0 defined near the point ^ satisfies for —> 0 
a relation 

^(<0 + /i) = «o + aiVl !+•••+ - DI + [a* + 

where oo, ai, ■ •, a* arc constants. Then a* is called the kth generalized de¬ 

rivative of yp at the point ta. It will be denoted by ^(jb)(fo)- The existence 
and finiteness of (/o) implies the existence of ^(*>(<0) and both numbers 
are equal. 

Another generalization of higher derivatives is based on the consideration of 
the symmetric differences 

A;i^(fo) = }p(h + h) jpik — h)y 

= 4>{U> + 2h) - 2^(<o) + Hto - 2A), 
aJV'«o) = Hto + m - Z4>{to + h)+ Zyl^ito -h)- - Zh). 
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If Ah^(to)/i 2 h)^ tends to a limit as -4 +0, this limit is called the Aiih sym¬ 
metric derivative of ^ at the point . We shall denote it by Clearly, 

Dk\l/(fo) exists and equals ^<*>(<0), if the latter number exists. 

It is a simple matter to prove (see [ 3 ]) that if /b is a positive even integer, 

and if the characteristic function <p{t) has at ^ = 0 a finite symmetric derivative 

/ +• 

X* dF{x) exists, and its value is 

00 

/ +ao 

x^dF(x) obviously implies (for k even) the 

00 

existence and continuity of (t) for all and in particular at the point ^ = 0. 

In order to obtain an extension of Theorem 1 to the case of derivatives of 
odd order, we have to generalize the notion of smoothness. We shall say that 
a function satisfies for t = condition Sk,{k — 1,2, • ‘ •),ii 

ArV(^) = o(h^) as +0. 

For A; = 1, condition Sk is identical with smoothness at to . Clearly, if ^(jt)(/o) 
exists, ^ satisfies condition Sk at to . 

Theorem 2. Suppose that k is a positive odd integer, and let v>(0 Ihe char¬ 
acteristic function of a distribution function F{x). If ^ satisfies condition Su 
at the point 0, a necessary and sufficient condition for the existence of Dkipiff) is 
the existence of the symmetric moment 

( 6 ) f x^dF(x) == lim xUF(x) 

J-00 x~*+«o J—x 

whose value is then equal to i '^Dkipiff), In particular, the existence of ^(*>(0) 
implies that of (6). 

The proof of Theorem 2 is analogous to that of Theorem 1. Let G{x) and 
H{x) have the same meaning as before. Since fc + 1 is even, condition Sh 
at the point / = 0 gives 

Ai+V( 0 ) = C" (c’'" - c-'^^Y^UF{x) = (sin xh)^^^ dF{x) 

J-OO •'—00 

= j["(8inxA)*+‘rf(7(a:) = o{h*), 

SO that 


(7) 


pl/A 

/ (sin d0{x) = o(/i*) 
Jo 

fUh 

J x^^UG(x) = o(0 

fUh 

/ dGix) = o{hh. 

Jl/2h 


( 8 ) 
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On the other hand, 

x'‘dF(x) = xUHix) 


m” 


J ^Hh fito 

+ / = Ah + Bh, 

0 Jl/h 


say. Here 


^00 r f,2lh fHh “I 

\Bh\<h-^ dG(x) = / + / + • • • 

Ji/h L'^i/h hih J 

= + • • * J = 0{l)y 

by (8). Since 

C'^y=+^^“’^1“=u+c>(«)i‘=1+ o(u) 

for small w, we immediately obtain 

pllh pllh 

Ah - / x'^dHix) = / dG(x) = o(l), 

Jo Jo 


by ( 7 ). Collecting the results, we see that 


Aim _ /•*'* 

^ & ( 2 /t)* I Jo 


x'‘dHix) = 


_tAU(0) 

* (2A)* 


A 1/A 

/ = o(l), 

J-i/h 


which completes the p;'oof of Theorem 2 . 

One more remark. By Theorem 2, the existence of the first moment is equiv¬ 
alent to the existence of the first symmetric derivative 


= lim;i-.o[^W - ip{-h)]/2h. 

In Theorem 1 we have a corresponding result for ordinary first derivative 
V>'( 0 ) = lim^-oo [<p{h) - AO)]/h. 


There is no discrepancy here since at every point where (p is smooth the two no¬ 
tions of derivative are equivalent. 
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A LOWER BOUND FOR THE VARIANCE OF SOME UNBIASED 
SEQUENTIAL ESTIMATES 

By D. Blackwell and M. A. Girshick 
Howard University and Bureau of the Census 

Consider a sequence of independent chance variables Xi ^ X2 r • with identical 
distributions determined by an unknown parameter 6 . We assume that E Xi = 6 
and that Wjc = xi • • • + jc* is a sufficient statistic for estimating 6 from 
xi f • • • , Xk • A sequential sampling procedure is defined by a sequence of 
mutually exclusive events Sk such that Sk depends only on , • • • , a;* and 
'S P{Sk) = 1 . Define W = Wk and n = k when Sk occurs. In a previous paper 
by one of the authors [1] it was shown that if Sk = W* C(/Si + • • • + &-i), 
(where C(A) denotes the event that A does not occur), the function V{W,n) = 
E(xi I W, n) is an unbiased estimate of and <t^{V) < a^{xi). It is the purpose 
of this note to obtain a lower bound for <r*(F). Our result is: 

Theorem i. o-^(y) > . 

E(n) 

We remark that the lower bound is actually attained in the classical case of 
samples of constant size AT . For in this case, (see [1]), V = E(xi | Wn) = Ws/N. 
In fact we shall show that in a sense this is the only case in which the lower bound 
is attained. 

The proof of Theorem I depends on certain properties of sums of independent 
chance variables. These, formulated more generally than is required for the 
proof of Theorem I, are given in 

Theorem n. Let ri , 0:2 , • • • he independent chance variables with identical 
distributions, having mean d and variance &^{x\). Let furthermore {/Sf*} be any 
sequential test for which E{n) is finite. Let W = Xi A- • • • + Xfc when n k. 
Then 

(a) a\W - Bn) < (xi) Ein), 

(b) If (T^(n) is finite, the equality sign holds in (a). 

(c) E[xi{W - Bn)] = <r\xi). 

Proof op (a). Write y, = x,- — B, and define F = yi -f • • * + y* when 
n = fc. By definition, 

(1) a^(W — Bn) = f (yi ‘ ‘ ~h yib)^ dP . 

fc-i Jsk 

To prove (a), we must verify that the series on the right of expression (1) con¬ 
verges and has sum <a^(xi)E(n), Now 

E f (vt+ ••• + y>:ydP 

jb.l Jsk 

< S /* (yi + ’ ‘+ y*)* dP + f (yi + • • • + vn)^ dP 

kll Jsk 

= 53/* yldP -jr 2 ^ f yk(yi + • * -f y*-i) dP. 

JU.1 Jn:&h fc-2 


( 2 ) 
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Since the event { > k] is independent of yk , each term in the second sum 
vanishes and the first sum becomes 

E f yldP = <rVi) E F{n>k\ 

kmml 

= <r*(a:,)[P{n = l| + 2F{n = 21 + • • • NP{n = iV| 

+ NP{n > N]] < c\xi)E(n). 

This establishes Theorem 11(a). 

Proof of Theorem ii(b). Write Zi = | y, | and let Z — zi + • • • -f 2 * when 
n = k. From (a) it follows that <r^[{Z — nE(Zi)] is finite. If in addition, 
c^(n) < 00 then E(Z^) < «. Thus the series 

(4) Ef (a+ ••• = E f ZiZidP 

kmml JSli l^t,jSfc<00 JSk 

converges, so that the series 

(5) E f ViVidP 

converges absolutely. The terms of the latter series may be arranged to yield 
(A): E f (vi + • • • + dP = <r* (JT - On) 

*-l Jsk 

or to yield 

B: z f j/Up + 2 E f yk(.yi+ ■■■ + yk-i) dP = c\xi)E(,n). 

All A;-2 J[n^k} 

This proves Theorem 11(b). 

Proof of Theorem n(c). It follows from Theorem 11(a) that Exi(lV — &n) 
is finite. If we show that 

(6) EiW — dn\ Xi) = xi — e, i.e, EiY | yi) = , it will follow [1] that 

(7) E{Xi.{W - (9n)l = P[xi(a:i - «)] = <r^a;i). 

To verify (6), it is sufficient to show that if /(xi) is the characteristic function 
of an event depending only on Xi (i.e. /(xi) = 1 when the event occurs, /(xi) = 0 
otherwise) 

(8) Eijy,) = E(SY). 

Write <#>i = 0, = /• ( 2/2 -f • • • + 2 /t), i > 2. 

Then it easily verified that 

(9) E{<f>j 1 Xi, • • • , x<) =0,- forj > i 

% 

E4>i < 23 12/* I 


ao) 

( 11 ) 


£?(<..) = 0 . 
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Hence it follows [ 2 ] that £0 = 0 where 4 > = 4>i when n = i. In our case 0 = 
Sy — fVi, and £0 = 0 yields (6). This completes the proof of Theorem II. 

Proof of Theorem i. In [1] it is proved that E(xi(W — en))= E[V{W — Bn)]. 
Hence employing Theorem II we get 

( 12 ) Axx) = EmW - Bn)] = a{V)a{W - Bn)p 

where p, (0 < p < 1 ), is the coefficient of correlation between V and W — On. 
Substituting for <r(W — dn) we get 

<r*(a:i) < (r(V)<T{xi) \/^(n) p 


Solving for a{V) we finally obtain 


( 14 ) 


AV) > 


^ Axi) 
- £(») 


which proves Theorem 

If <r*(n) is finite, the equality sign in ( 14 ) will hold if and only if p = 1. We 
shall now prove the following. 

Theorem m. Let N he the minimum value of n for which P(n — N) 9^ 0 , 
TheUj a necessary and sufficient condition that p = \ is that P{n = A^) = 1. 

Proof. The sufficiency of this condition follows from the fact that if 
P(n = AT) = 1, y = W/N, To prove the necessity of this condition, we 
observe that if p = 1, F is a linear function of W — nB. That is, 


( 15 ) 


V = a{W - nS) + / 3 . 


Now, since EV = 6 and E{W — nB) — 0 , it follows that ^ B. Also, since 
by hypothesis a^{V) = a^{xi)/E{n) and a^(W — nB) = a^(xi)E{n)y it follows 
that a = 1/E(r2). Hence the estimate V is given by 


( 16 ) 


= 


W -nB 
E{n) 


+ B, 


* Under certain regularity conditions Cramer has obtained the inequality 

where f * f(a 5 , 0) is the density function of x ([3], p. 475). Thus with the same regularity 
conditions, our inequality yields 

<r'{V) ^i/mn)E(^-^y. 

which is a special case of the results presented by J. Wolfowitz in this issue of the Annals. 
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Let N be defined as above. We note that N < ^ since by hypothesis E{n) < . 

Let Vjf he the estimate of B when the sequential test terminates with n = N, 
Then Fjv = W/N. Substituting this value in ( 16 ) we get 



We exclude the trivial case where W = NB. Then ( 16 ) yields E(ii) = N, 
That is P(n = W) = 1. This proves the theorem. 

We remark that N may be a function of B but for a fixed B, n = iV is fixed 
when p = 1. 
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AN EXTENSION TO TWO POPULATIONS OP AN ANALOGUE OF 
STUDENT’S t-TEST USING THE SAMPLE RANGE 

By John E. Walsh 
Princeton University 

1. Summary. The modified f-test considered by Daly^ (see [1]) is used to 
develop one-sided significance tests to decide whether the mean of a new normal 
population exceeds the mean of an old normal population having the same 
variance. Significance tests are also developed to decide whether the mean of 
the new population is less than the mean of the old population. These tests 
require very little computation for their application and are approximately as 
powerful as the most powerful tests of these hypotheses. 

2. Introduction. Let n , • • • , rn , (n < 10 ), be independently distributed 
according to a normal distribution with zero mean and unit variance. Let r^u) 
denote the uth largest of the r's. Then Daly has shown how to determine 
numbers ga such that 

Pr[f/{j(.) - n,)) > jrJ = a 

( 1 ) 

• Pr[f/(r(„) - r(i)) < -p„] = a. 

This note will use these relations to develop easily applied signifieance tests to 
decide whether the mean v of a new normal population exceeds the mean m of 


‘ This problem is also considered by Lord in [2J. This note was in proof when [2] appeared. 
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an old normal population with the same variance. Significance tests are also 
developed to test v < fi. The simplest case considered is that of testing a new 
sample value x on the basis of n past sample values 2/1, • • * , 2/n • Then the 
significance test at significance level a to decide whether v exceeds /x consists in 
accepting v > tiU • 

*>§ + {/. Vn + [Viv) - yail, 

where 2 /(m) is the uth largest of , • • • , . 

The significance test oi v < fx consists in accepting 1/ < /x if 

^ <y - Vn + 1 [y(») - yw\- 

These tests are generalized to the case in which x is the mean of a sample of 
size r from the new population, each of 2/1, • • • , i/n is the mean of a sample of 
size s from the old population, and z is the mean of a sample of size t from the 
old population. Then the tests at significance level a take the form 

Accept V > IX if X > {I - Ci)y + Tiz -f Qalyin) — 2/(1)]; 

( 2 ) 

Accept V < IX if X < (I - Ci)y + Ciz — Qalyin) — 2 /a)]> 

where Ci is a given constant which is selected by the person applying the test. 
The introduction of the terms z and Ci allows less reliable past information to 
be utilized by lumping it together in the z term and using the constant Ci to 
weight this information according to its relative importance with respect to 
the 2/'s. 

The power of test (2) is compared with that of the corresponding Student /-test 
for the case Ci = 0 and n < 10 . In this comparison the quantities x,yi y • • • , 2/n 
are considered to be the given sample values which are used for the test, that is, 
the quantities from which the means x, 2/1, ' • * , were formed are not given. 
It is found that the power of the Student /-test is only slightly greater than that 
of the corresponding test ( 2 ). For the cases considered, however, it is well 
known that the most powerful test of > m using the quantities a;, 2/1, • • • , 2/n 
is the appropriate Student /-test. Similarly for testing v < ix. Thus the tests 
(2) considered are approximately as powerful as the most powerful tests of 
V > IX and v < ix which use a:, yi, • • • , yn . 

Examination of ( 2 ) shows that the amount of computation required for the 
application of one of these tests is small. Consequently the tests ( 2 ) have the 
desirable properties of being easily computed and nearly as powerful as any 
tests which could be used for the given hypotheses. This suggests their use in 
repetitive testing procedures which are concerned with the testing of the mean 
of a new sample on the basis of the means of previous samples. 

3 . Statement of tests. In this section three significance tests of increasing 
generality are stated. It is to be observed that each test is a particular example 
of the test following it so that tests (A) and {B) are special cases of test (O. 
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The reason for stating tests (A) and (B) is that these tests have a much simpler 
appearance and will cover most cases of practical application. 

(^). Let each of x, 2/1, * * • , 2/n represent the mean of a sample of size r; let 
the values of the sample whose mean is x have the distribution N{v, and the 
values of the samples whose means are • , yn have distribution <r^), 

where the notation A^({, <r^) denotes the normal distribution with mean f and 
variance Then the significance test oi v > n &t significance level a is 

Accept v> n if X > y + g, ^ [vm - 

The significance test to decide whether v < n'is 

I 2 

- [2/(«) - 2/(1)]. 


(B), Let X equal the mean of r sample values from N{v, a^) and each o^ 
2/1 j * * * > 2/« CQual the mean of s sample values from The significance 

test for V > n at significance level a is 


Accept V > n if X > y + g, 


+j[2/u) 


The test of p < pis given by 


Accept V < p if X < y 


~ + - t2/(n) ~ 2/(1)]. 

r s 


(r). Let X equal the mean of r sample values from N(py each of yi, • • • , 2/n 
equal the mean of a sample of size s from Nip, o-^), z equal the mean of a sample 
of size t from N(p, <r*), and Ci be a given constant value. Then the significance 
test of V > M at signifioance level a is 
Accept V > pif 


X > (I - Ci)y -f- Tiz + [2/(n) “ 2/(i)]<;a 


Ir / 


(1 - C iy \ 


The significance test to decide whether p < p is 
Accept p < pif 


r'^ti 


a: < (1 - Ci)y + Ci 2 - [yin) - yw]ga ■ /(- + 

Values of for a = .05 are given in Table I. These values were listed by 
Daly in [1].* 


‘Values of ga for a « .05, .025, .01, .005, .001. and .0005 are listed in Table 9 of [21 for 
sample sizes from 2 to 20. 
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4. Derivation of tests. As tests (A) and (B) are particular cases of test 
{C)t it is sufficient to derive test (C). 

TABLE I 


Estimated Values of g.o^ 


n 


3 

.882 

4 

.526 

5 

.385 

6 

.309 

7 

.260 

8 

.227 

9 

.202 

10 

.183 


liCt the quantities x',y[, ■■■ ,y'n, z' l)e defined by 
, (a: — k) \/r j (y. — m) V» 

X* = ^-, =--, 

(f = 1, . 


a 

m) V/ 

or 

. 



Then x', t/i , • • • , y'n , are independently distributed according to N{Oj 1). 
Define 


r« = ^ ~ 2 /* + + ^2Cz^ , (w - 1, • • • , n). 

It is easily verified that 

E(r.) = 0 , E(rl) = [K? + (1 + C^)Kl - 2 K, + n] 

E{rur^) = [(1 + - 2Ki + »], ' (w w). 


Thus, if Ki and Ki satisfy the equations 

< 3 , + + 

(1 + e)K\ - 2 Ki + n = 0 , 

the r„ will be independent of y. when y = v. Also they will be independently 
distributed according to iV( 0 , 1 ). 
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Rewriting the r„ in terms of x, 2 one obtains 

( 4 ) ru = — 2 2/< + ^ *')J- 

Using ( 3 ) the mean of the is found to be 
^ K%\/r\ 


Kia 


■|^® - \/^V + ^ \/l^ - (v - m )]- 


Let r(u) denote the ?ith largest of n , • • • , r„ . Then from (1) 
a = Pr[f/(r<n, - r(.,) > ga ] = (:r - + C^y 

+ C 2 - (v- fl)^ ^ (.y^n) - Vd)) > fl'.J 

It is easily proved from ( 3 ) that 

^ . A + cW„ , {Vr+cVin 

K,Vr -T-V+-^(rTW~)' 

Choosing the positive sign, putting C = — Ci, and letting m = y one obtains 


Pr 


X > (1 — Ci)y -i- Cl z 


+ [^(n) “ VixA^t 




verifying the first part of test (C). The second part of test (C) is verified by 

Ki 


choosing the negative sign for ^ 
the second part of (1)). 


-- (or by repeating the above argument using 


6. Power comparison with t-test. Let x, 2/1, • • • , t/„ satisfy the conditions 
of test (B) in section 3 . Then Student’s / using x, 2/1, • • • , ?/„ is given by 

t = [a: - y - (y - m)1 / n - 1 ~ 

The Student /-test based 'on this value of t furnishes the most powerful test of 
V > fi (and V < n) using x, 2/1, • • • , ?/« . The purpose of this section is to show 
that test (B) has approximately the same power as this Student /-test for n < 10 . 

Daly has shown (see [1]) that if n , • • • , r„ are independently distributed 
according to iV(f, a*), then the test based on 

(f - f)/(r(H) - r(i)) 
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has approximately the same power for testing f > 0 (and f < 0) as the corre¬ 
sponding Student ^test based on 

(5) t = - pVn(n - 1) 

l/z (n- - f)* 

for n < 10. 

Using the notation of section 4 let 

»•« = ^ -'^yi + Kt a;J, (m = 1, • • •, n). 

Ki 

where > 0 . Then from consideration of ( 4 ) with C = 0 it is seen that the Vu 
A2 

are independently distributed according to Ar(f, <r*), where f equals a positive 
constant times (v — /i)- Following the derivations in section 4 with C = 0 , 
it is seen that the test of { > 0 with this particular choice of the r„ is identical 
with the test of > /i given in (B) of section 3 . Similarly the test of f < 0 is 
identical with the test (B) oi p < fi. Thus the test (B) has approximately the 
same power for testing v > n (and v < m) as the Student ^test based on the value 
of t given in ( 5 ) if n > 10 . Replacing the r„ in (6) by their values in terms of 
2/1 > * * • > 2/n > w, r, and s, it is found that (6) becomes 

n — 1 
\r 718 / 

This proves that test (R) is approximately as powerful for testing v > n and 
V < HBB the most powerful test based on the quantities x,yi, • • • , ?/«if < 10. 
As test (A) is a particular case of test (R), these results also apply to test (A). 
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ON THE NORM OF A MATRIX 

By Albert H. Bowker 
University of North Carolina 

In studying the convergence of iterative procedures in matrix computation 
and in setting limits of error after a finite number of steps, Hotelling [1] used 
the square root of the sum of squares of the elements of a matrix as its norm. A 
wide class of functions exists which may be employed as norms in matrix calcula¬ 
tion and substituted directly in the expressions derived by Hotelling. The 
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purpose of this note is to make a few general remarks about this class of functions 
and to propose a new norm which appears to have some value in computation. 

A function 0(A) of the elements of a real matrix A may be termed a legitimate 
norm if it has the following four properties: 

(1) 0(cA) = I c I 0(A), c a scalar; 

(2) 0(A B) ^ <t>{A) + 0(7^), if A -f is defined; 

(3) 0(A^) ^ 0(A)0(J^), if AB is defined; 

(4) 0(c,j) = 1, \vhere e^j is a fundamental unit matrix 

whose elements are all zero except the one in the ?th ro>\ and ^th column, whose 
value is unity. These four conditions are identical Avith the first four axioms 
of Rella [2], who has shown them to be independent. Properties (1), (2), and 
(3) are used directly in investigations of convergence and error, but the im¬ 
portance of property (4) is indicated by some of its immediate consequences. 
Clearly e^Acj = , Avhere c* is a fundamental unit vector. From (3) and ( 4 ) 
it follow s that | | ^ 0(A) for all i and^ and we have that 

(5) max(.j) I a., I ^ 0(A). 

Thus 0(A) has the useful property that the norm of a matrix of errors exceeds 
or equals the maximum possible error. Since 0(A”*) ^ 0”*(A), it folloAvs from 

(5) that the elements of A” wall tend to zero as m increases if 0(A) < 1, a result 
which is useful in establishing convergence. Also 0(A) ^ 0. 

One further consequence of (1) to (4) is of interest. Suppose A is a square 
matrix and let X be any of its roots. Then there exists a non-null vector a* 
such that Ax = Xx. Now 0(Xx) = X0 (t) ^ 0(A)0(r) and we have 

(6) X ^ 0(A). 

Thus, every legitimate norm is an upper bound to the characteristic roots. 
Clearly many functions exist which satisfy (1) to (4). The norm used by 

Hotelling is N(A) = • A new' norm which may have some value is 

obtained as follows: 

(7) ^(A) = maX(.)/?,(A) 
where 

R^(A) = Z I I • 

i 

Clearly R{cA) = | c j R(A). To show that R satisfies (2), consider 

RM + B) = Z I + *»« I ^ Z I «» I + E I 6.; I ^ R(A) + R{B). 

1 J i 

Since the above inequality holds for all i, 

R{A +B) S RiA) + R{B). 
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Now AB *= II S o>icbaj II 

a 

and 

Ri{AB) =E I E I ^ E E I I • I 6<.i I 

i a i a 

^ E I Ofa I R«(.B) g BiB)R(,A). 

a 

Hence R{AB) ^ B(A)R(B). Clearly R(e{j) = 1. Similarly it may be shown 
that C(A) = max(j) ^ | aij | also satisfies the conditions of a norm. 

i 

Since the convergence of an iterative procedure is often proved by the norm 
being less than one, since the norm appears in the upper bound for the error 
after a finite number of iterations, and since the norm of a matrix of errors is 
taken to indicate the magnitude of the errors, a reasonable method of choosing 
among several available legitimate norms is to select the smallest. It is natural 
to inquire whether an optimum norm in this sense exists; that is, is there a 
function <i>*{A) such that <f)*{A) possesses properties (1) through ( 4 ) and such 
that <f)*(A) ^ 0(A) for all other 0(A) satisfying these conditions. Assume such 
a 0*(A) does exist. Clearly 0*(A) = 0*(A'), as, if either exceeded the other, 
the smaller could be taken as 0*(A). Let A^.be the largest root of A A'. Then 
by (6) 

a' ^ 0*(AA') ^ <f>*\A) and A ^ 0*(A). 

But Rella [ 2 ] has shown that A possesses (1) to ( 4 ). Thus 

0*(A) = A. 

But, for a row vector, C(A) ^ A. Consequently, no minimal norm exists. It is 
interesting to note that a worst norm does exist, namely P(A) = ^ | Oij [ . 

Since A = ^ eijaij , 0(A) ^ P(A). Clearly P(A) satisfies (1) to ( 4 ) and hence 

is the worst possible legitimate norm. 

In practical computation, the choice so far is between N{A) and /?(A) (or 
C(A)). No general inequalities exist and it would probably be advisable to 
compute both. R{A) may be less than Ar(A) and indicate convergence when 
N{A) fails to do so. Often R{A) may be computed visually and convergence 
proved without computing the sum of squares of the elements. 

The functions N{A) and /?(A) may also be useful in finding a simple first 
approximation to A~\ A sufficient condition that Hotelling*s iterative method 
for finding the inverse of a matrix A will converge is that the roots of 
Z> = 1 — ACo be less than one in absolute value where Co is a first approximation 
to A~K If the iterative procedure is to be carried out by a fully automatic 
computing machine such as the one described by Alt [ 3 ] it may be advisable to 
start with a rather poor first approximation which is easy to construct. If A 
has positive roots and if M is any upper bound to these roots and if Co is a matrix 
with diagonal elements equal to 1 /M and zeros elsewhere, the iterative procedure 
will converge but the norm of D will not necessarily be less than one. From 
(6), any legitimate norm may be taken as M. 
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Finally, it is interesting to point out the relation of this note to some work on 
the problem of finding upper bounds to the roots. In fact, the inequalities 
X ^ N(A) and X ^ R(A), which are consequences of (6), are Theorem 2 of 
Farnell [ 4 ] and Theorem 3 of Barankin [ 5 ] respectively. 
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DEFINITION OF THE PROBABLE DEVIATION 

By M. FniicHKT 

Faculty of Science^ University of Paris 

The probable deviation has recently been defined by E. J. Gumbel [1], [2] 
as the smallest of the intervals corresponding to the probability It so hap¬ 
pened that the author was led to an equivalent definition starting from a general 
idea which may be applied to absolutely general cases and which, for this reason, 
might be of interest. 

In recent years, the author has been occupied with a study of random ele¬ 
ments of any nature (curves, surfaces, functions, qualitative elements), a study 
whose future seems promising, [ 3 ]. I gave a definition of the mean of such an 
element expressed by an abstract integral which, however, is only defined if the 
random element is situated in a metric vectorial (Wiener-Banach) space.^ But* 
a still more general definition is valid if the random element is placed in any 
metric space. It consists of taking, as mean position of the random element X, 
a fixed (non-statistical) element h = X such that the function of a which rep¬ 
resents the mean M{X, a)* of the squared distance of X to the fixed element o, 
is minimum for a = b, (In the case where X and a are numbers, and where 
M(Xf is finite, we know that this minimum is reached and that there is one, 
and only one, determination 6 of a). This definition has the advantage of also 
defining the equiprobable position of X. This is a fixed element c = ^ such 
that Af(X, o) is minimujn for c = a. (If X and a are numbers, we know that 
this minimum is still reached, but may be so reached by several values of X). 

Since reading GumbePs paper, a still more general definition suggested itself. 


* For the definition of metric vectorial spaces see [4]. 

* See Note 2, p. 503 of 14]. 
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The expressions M(X, a) and "v/ M(X, a)* themselves may be considered as 
distances, but as distances of two random elements taken together. To each 
of these distances corresponds as minimum, when a varies, a different ‘‘typical” 
function S or X • • •. Thus, without supposing anything about the space 
into which the different trials place X, we assume that we have defined a “de¬ 
viation” of two random elements X, Y taken together. We represent this 
function of two random variables by (([X], [F])), a notation which differs from 
the representation of the distance (X, Y) of the two positions X and Y with 
respect to a single trial. The lower boundary of the deviation (([X], [a])), a 
function of a, which is reached for a = X defines a “typical” position X. More¬ 
over, the value of this (([X], [X])) may be considered as a measure or, at least, 
as a numerical ranging point of the dispersion of X. 

Let us abandon these generalities. They hold especially if the element X 
is a real valued random variable. Among the possible and reasonable* expres¬ 
sions for the deviation (([X], [a])) of the numerical variate X from a fixed number 
a, we may use the equiprobable value of [ X — a | which may be called the equi- 
probable deviation of X from a. Thus we have, on one side, a new “typical 
value” of X which will be a value of a such that the equiprobable deviation of X 
from a is minimum, and a new measure of dispersion which is the value of this 
minimum and which might be called simply the equiprobable deviation of X. 

In the case where X has everywhere a continuous and finite density of prdb- 
ability w{X) we find, as typical value, what Gumbel calls the “midvalue” 
and represents by and, as equiprobable deviation, what Gumbel calls the 
“probable deviation” and represents by f. 

We may also consider the discontinuous case, which was given as a problem 
to candidates of the “Certificat d^Etudes Supdrieures de Calcul des Probabilitds, 
Option Statistique Mathdmatique, Session May-June, 1944 .” They had to 
solve various questions of which I cite the beginning below: 

“Consider n real numbers xi ^ X2 ^ ^ Xn and represent, by Eo , a median 

value of the deviations | Xa, — a | of the numbers Xk and o. If a varies, Ea has 
a minimum E which is reached by one or several values A of a. 

1 ) Explain, in a few words, the meaning of the values E and A. 

2 ) For simplicity’s sake, suppose that n is odd (n = 2 r + !)• How should 
E and A be calculated practically? (To find the answer, investigate first how 
Ea varies if a varies only slightly). 

3 ) In the case where n = 4 s -f 3 (s is an integer equal to, or larger than, zero) 

show that E ^ ^ 

where Qi = x »^\, qz == Xn-t 

The study of this new typical value and of this new equiprobable deviation 
has the advantage that their determination is very rapid and requires hardly 


* See the Remark at end of note. 
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any calculations. However, we have to note an important inferiority of the 
equiprobable deviation of X compared to the mean and the standard deviations 
of X. If one or the other of the last two deviations is zero, X is a fixed number 
(except for the case of the probability zero). This property seems requested 
by the intuitive meaning which we attribute to the dispersion, and to every 
measure or any mark of it. Now, the equiprobable deviation lacks this property. 
If, for instance, X has only three values: 0, 2,1, the first two with the probability 
0 . 249 , and the last with the probability 0 . 502 , the equiprobable deviation of X 
will be zero, whereas X will be equal to its typical value 1 only with a prob¬ 
ability of 0 . 502 , and not with a probability equal to unity. The same holds 
for any distribution for which there is a point with probability exceeding J. 

Remark, The definitions of the mean and of the equiprobable position become 
meaningless in the case that il/(X, a), or af, is infinite. However, we 

succeeded in surmounting the difficulty, and to reach definitions which arc valid 
even in this case. If X is a number, the new definitions become equivalent to 
the classical definitions of the mean and equiprobable value. The proofs arc 
given in two recent articles [ 5 ], [6]. 
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THE GENERAL RELATION BETWEEN THE MEAN AND THE MODE 
FOR A DISCONTINUOUS VARIATE 

By M. Fr^chet 

Faculty of Science^ University of Paris 

Dr. Gumbel has pointed out that one of the author^s arguments employed in 
several particular cases (see [1]) can be employed in a general case which includes 
them and leads to the following result: If a statistical variate R has only positive 
entire values differing from zero, and if its mean value R is smaller than, or 
equal to, unity, the same holds for its equiprobable value R and its mode R. 
There are two generalizations of this result which might be of interest: 

1 ) On the one hand, the author has shown [2] that, if a variate R can only 
have values (entire or not) equal to, or larger than, zero, its equiprobable value 
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R is, at most, equal to twice its mean value R, and the inequality R/R ^ 2 
cannot be improved which means that the upper boundary of the first member 
is exactly equal to (and not less than) twc). The equality is reached when R 
has only two values of equal probability, one of them being zero. 

2 ) On the other hand, if R is an integer positive variate equal to, or larger 
than zero, it can be proven that, R ^ a, we have 

( 1 ) li ^ ^ . 


Here, R and R stand for the mean and for the mode of R respectively, and ct is 
a positive integer differing from zero. For example: if R is the number of rep¬ 
etitions of an event with probability p, we have, for n trials, R = np, whence, 
if a is the first integer number equal to, or larger than, R we have the inequality 
( 1 ) for the most probable number of repetitions. Naturally, this inequality 
only has an interest if the second member of (1) is smaller than n which means 
that 

<x{(x -j- 3 ) <! 2a. 


This presupposes 


2 n > np{np -f 3 ) 


r? < 


2 ~ 3 p 


and, since n must be positive, 


p < I- 


To prove the ineciuality (1), let us write u, for the probability that R 
We have 


whence 

( 2 ) 

Let the mode be 

then 


OJv = 1 ; VUSy = R S 


52 (a — v)<ap ^ “ a)wp. 

0 «+i 


R = 0 


wfl ^ "f ; I* = i> 2, 
and the first member in (2) is bounded by 


^ £(« - .K. 
0 


( 3 ) 


2 
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Now, either a < fforp S a. In the first case the second member in (2) leads to 

00 

(4) Z) (>' “• «)«*>*' ^ CS — a)u)0 

a+l 

since the second member in (4) is one of the terms occurring in the sum. The 
same inequality holds in the second case, jS ^ a, hence it holds generally. It 
follows from (2), (3), and (4) that 


a(a + 1) 
2 


^ 0 ^ — odcjfi. 


00 


The probability wfi is certainly different from zero, since X) “ 1* 

0 

quently 


/3 — a 


2 


Conse- 


or 

s ^ 

^ - 2 

as stated in (1). 

The equality in (1) is possible only if, from (3), 

a(<u)fi — a>o) + (a — — wi) + • - • + (oj^ — Wa-i) = 0 

and from (4) 

CUa+l + 2cJa+t + • • • + (iS ~ + • * * = (jS — a)(afi 


whence 

(6) Wo = 0)1 = • • • = W/S = • • • = Wa-l 

and 

(50 W«+i = Wa+* = • • • = 0. 

The existence of the exceptional case proves that the inequality (1) cannot 
be improved by replacing the second member by a smaller function of a. In 
the exceptional case, the only possible values of R are 


= 0, 1, 2, •••, a - 1, a, jS, 


and all values, except perhaps a, are equiprobable. The probability w« may 
be, but need not be, equal to cop . 

Moreover 


. ^ + 3) 

P * -3?- 


^ a 


( 6 ) 
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and ^ = a is possible only if a = ^ = 0 whence, from (5), = 0 except for 

V = 0 which means that R only has one value equal to zero. Except for this 
trivial case, we have in the exceptional case > a, and there are a + 2 possible 
values for R, Then we must have 


whence 


^ «a; ^ = 1 

0 

(a 4" + Wa = 1 


and, from (5), 

a ^ R = OJfi ^ V + + OMa = 0)0 I -^- - 4* - 1 + 

= a((a + l)o)i3 + 0)a) 

whence 

(7) fj = a. 

From 


OCOa 


1 = (a + 1)«^ + Wa ^ (« + 2)«« 

follows 


( 8 ) 


0 )a ^ 


a -f" 2 * 


0)0 


1 — Wa 


These conditions (5), (5'), and (7) are necessary and sufficient for the existence 
of the exceptional case. 

If the equality in (1) is excluded, the mode 0 and the smallest integer number 
a which is equal to, or larger than, the mean, are related by 

(9) ^^«(a^_l^ «* + 3 a^2 

2 2 


As shown before, this general inequality, valid for any discontinuous variate, 
which can assume only non-negative integer values, cannot be improved without 
assuming specific properties of the distribution. 
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NOTE ON DIFFERENTIATION UNDER THE EXPECTATION SIGN 
IN THE FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS 


By T. E. Harris 
Princeton University 


Let z be any chance variable and zi , 22 , 23 , * • * a seciuence of independent 
chance variables, each with the same distribution as z. Let Zn — Z\ + Z2+ * * * 
+ Zjf . Let 4>{t) = Ee*^ for all complex t for which the latter exists. Let <Si, 
S 2 i • • • be a sequence of mutually exclusive events such that depends only 

00 

on 2i, 22 , ' -Zj, and 23 = L Let the chance variable n be defined 

as n — j when Sj occurs. Blackwell and Oirshick [1], j>;cneralizing a result 
of Wald [2], showed that if there is a positive constant M such that 

(1) |Za 1 <Myvhcnn>N 
then the identity 

(2) E\e^-\<f>(t)ri = 1 

holds for all complex t for which <f){i) exists and | 0(0 | > L Wald [3] estab¬ 
lished conditions, including the existence of 0(/) for all real /, under which 
(2) may be differentiated under the expectation sign an unlimited number 
of times. 

Without assuming the existence of 0(/) for a real ^-interval the following result 
holds: If (1) is true and if -^( 2 *) and E{n^) are both finite, k a 'positive integer^ 
then 


(3) 

where i = \/ — 1 and s is real. Certain identities, obtained by differentiating 
(2) and plotting / = 0, can also be obtained from (3). For example, if En = 0, 
and if AV and Ez^ both exist then EZ\ = Ez^En. 

Let Ps = P(n < AT); p^ = P(n = N). Let Hij, Zi) and F{N, Z^) be the 
conditional cumulatives of Zj and Zn for n ^ j and n > N respectively. Now 
(2) was derived by Wald [2], p. 285, from a relation, valid whenever 0(0 exists, 
which in the present notation becomes 


( 4 ) E Pi r dHU, z) + 

/■■I *f—70 


(1 - p«) 


r dF(N, Z^) = 1. 

J_00 


Examination of Wald’s derivation of (4) shows it to be valid under the present 
hyjjbtheses. Now the finiteness of E{z^) clearly implies that of E(Z) | n = j). 
Also, since F(N, Zff) is constant outside the interval [—M, M], the integral 


£ 


Zs dFiNy Zjf) is finite. 


Hence we may set i = is in (4) and differentiate 
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k times, obtaining for all real s 


(5) 




+ 


(1 - -Pi.) E (*) ^ (iZ„)'-'e^«''dF{N,Z„) = 0. 


The derivatives of {<l>{is))~^ are sums of terms of the form Q{N) • {<t>(is))~^^'‘ 
times terms independent of N, where Q{N) is a pol 3 aiomial in iV^ of degree < k. 
For any r < k. 


lim I (1 — Pn)N^ I = lim 

N-*oo N-*oo 


t V, 

< lim 

00 

E 3* Vi 

J-A+l 

V-^oo 

J-AT+l 


= 0 , 


since En^ is finite. Hence lim (1 — Pif)Q{N) = 0. Because of (1) the inte¬ 
grals in the second term of (5) are bounded as A" —^ oo. Now set s = 0 in (5) 
and then let oo. Since 0(0) = 1, the second term of (5) approaches 0 

and the limit of the first term is just the left side of (3). 

For the case of a Wald sequential process, Stein [4J has shown that all moments 
of n are finite. In this case (3) holds whenever Ez^ is finite. 
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A UNIQUENESS THEOREM FOR UNBIASED SEQUENTIAL 
BINOMIAL ESTIMATION 

By L. J. Savage^ 

University of Chicago 

In a recent note [1], J. Wolfowitz extended some of the results of a paper by 
Girshick, Hosteller and Savage [2] on sequential binomial estimation. The 
present note carries one of Wolfowitz's ideas somewhat further. The nomen¬ 
clature of [1] and [2] will be used freely. The concept of “doubly simple region’* 
introduced in [1] and assumed there only in the hypothesis of Theorem 3, will 
here be shown to be unnecessarily restrictive. In so doing, we find that sim- 


^The author is a Rockefeller fellow at the Institute of Radiobiology and Biophysics, 
University of Chicago. 
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plicity is not only a necessary (cf. Theorem 4 of [2]) but also a sufficient condi¬ 
tion that i> be the unique unbiased estimate of p for a closed region. 

Lemma. If R is simple there is at most one hounded unbiased estimate of any 
given function of p. 

Proof. If the lemma were false, there would be a non-trivial bounded un¬ 
biased estimate of zero, i.e., m{a) such that | m{a) | is bounded by a constant 
m*, m{a) not identically zero and E(m{a) | p) s 0. 

(1) E(m{a) 1 p) = m(a)k(a)p^q' = 0. 


and mia) not identically zero. Since R is simple we may assume (much as in 
the proof of Theorem 6 of [2]) that we have a boundary point such that 
m{ao) 9^ 0, ao is below all accessible points of its own index and also below 
every other a for wffiich m{ct) ^ 0. Therefore 

(2) I m{ao) 1 k{fitQ)p^'' = | m{a)k(ot)p^ q"" \ < m* ^ k{a)p^ q" . 

v>yo v'>uo 

Let M denote the set of all accessible points and boundary points at which 
X < Xo and y = t/o + L There are at most xo points in A/, say /3i , • • • , j9« . 
Considering the w^ay in wffiich ao has been chosen, every path from (0, 0) to an a 
for which y > yo passes through or to at least one point of A/. Therefore when 

y > yo 

P{a I A/)P(Af) 

P(a I J/) E 
1 

E K0,)P(a I M). 

1 


(3) 


P{a) = k{a)p^q^ = 
< 


From inequalities (2) and (3). 


(4) 


I m(a„) I fc(ao)p‘'"9*'> < |E *08,)) E P{ct \ M) 

L 1 ) V>Vo 


But it is impossible that (4) should be satisfied for small p. 

Combining the Lemma with Theorem 4 of [2] we have the 
Theorem. A necessary and sufficient condition that p(a) he the unique proper 
{hounded) and unbiased estimate of p for a closed region R is that R he simple. 

The sufficiency part of this Theorem extends Theorem 3 of [1] from doubly 
simple regions to simple regions. 

The author is indebted to J. Wolfowitz for his valuable suggestions in connec¬ 
tion with the present note. 



ACKNOWLEDGMENT OP PRIOKITY 


297 


REFERENCES 

111 J. WoLFowiTz, “On sequential binomial estimation,” Annals 0 / Math. Stat., Vo\. 17 
(1946), pp. 489493. 

[21 M'. A. GiBBHiCK, Fredebick Mostellbr, and L. J. Savaqe, “Unbiased estimates ior 
certain binomial sampling problems with applications.” Annals of Math. Stal.^ 
Vol. 17 (1946), pp. 13-23. 


ACKNOWLEDGEMENT OF PRIORITY 

By H. E. Robbins 
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At the time of publication of my papers on the measure of a random set 
{Annals of Math. Stat., Vol. 15 (1944), pp. 70-74; Vol. 16 (1945), pp. 3^- 
347), I was unaware that the theorem on page 72 of the first paper, which 
affords a means of computing the expected value of the measure, had already 
been found by A. Kolmogoroff. {Grundbegriffe der Wahrscheirdichkeitsrech- 
nung, Ergebnisse der Mathematik, Berlin, 1933, p. 41). I wish to take this 
opportunity of acknowledging Kolmogoroff’s priority, which was pointed out 
by Prof. Henry Scheff6. 
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Presented on January 25, 1947, at the Atlantic City meeting of the Institute. 

1. A Test of Significance of the Coefficient of Rank Correlation for more than 
Thirty Ranked Items. Nilan Norris, Hunter College. 

Hotelling and Pabst {Annals of Math, Stat., Vol. 7 (1936), p. 37) have suggested the use 
of the Tchebycheff inequality as an approximation for testing the significance of the co¬ 
efficient of rank correlation in cases where the number of ranked items is too large to enable 
exact probabilities to be computed directly. A table prepared in accordance with this 
suggestion indicates that for values of the coefficient of rank correlation larger than .50 
there is a wide range of corresponding numbers of ranked items greater than thirty for 
which at least the five per cent level of significance is satisfied. 

For certain types of applications the conservativeness of the TchebychelT test may be 
a virtue rather than a limitation. 

2. A Generalized T Measure of Multivariate Dispersion. Harold Hotelling, 
University of North Carolina. 

The problem of combining errors in two or more dimensions to measure the accuracy of 
firing and bombing is similar to problems occurring in industrial quality control where 
different measures of quality are applied to the same article, and to problems in mental 
testing and other fields. If the covariances were known a priori, the solution optimum 
in certain senses, for a multivariate normal distribution, would be the use of x* ~ S2X,/x<a;y, 
where is the covariance matrix and X{ is the deviation in the ith dimension. Since 

the covariances must in all known practical cases bo estimated from a preliminary sample 
with (say) n degrees of freedom, x* may be replaced by T* = , where is 

the estimated covariance matrix. This is the same T introduced by the author in 1931 
as a generalization of the Student ratio t, and has the same distribution. Upon adding 
together the values of T* for different cases (e.g. for different bombs dropped with the same 
bombsight), a combined measure Tl of over-all excellence (e.g. of the bombsight), is ob¬ 
tained. To like x^, can be broken down into components meaningful with respect to the 
causal system, specifically in relation to possible sources of excessive discrepancy. Thus, 
if Xi is the /h coordinate of the centroid, or mean point of impact, of m bombs, we may 
write , TJ = TjJ — Tlf . Then To is a function only of deviations from 

the mean point of impact. Asymptotically (for large n), To, Tm and Td have the x dis¬ 
tribution -vidth m, 2 and m — 2 degrees of freedom respectively. But the untrustworthiness 
of the X distribution as an approximation is evident even with n as large as 266, for which 
case calculations have been made. The exact distributions of To and Td are ascertained 
when the number of variates p is 2, and the probability integrals are expressed as linear 
functions of two incomplete beta functions. In fact, Tq/M equals the sum of the roots 
of a detcrminantal equation of the form | A — X/3 | = 0, where A and B are sample covariance 
matrices with n and m degrees of freedom respectively, and a similar relation holds for Tj 
with m replaced hy m — 2. To and Tm have the distribution published in 1931, with prob¬ 
ability integral expressible in*tcrms of a single incomplete beta function or the variance 
ratio distribution. It is shown that such parameters as the circular mean deviation are 
best estimated with the help of the T measures, not directly by averaging individual cir¬ 
cular deviations. 

3. Asymptotic Properties of Maximum and Quasi-Maximum Likelihood Esti¬ 
mates. Herman Rubin, Cowles Commission for Research in Economics. 
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The results of J. L. Doob (Trans. Am. Math. Soc.j Vol. 36 (1934), pp. 759-775) on con¬ 
sistency of maximum likelihood estimates, are generalized and extended to arbitrary mea¬ 
sure spaces. In some special cases, results on asymptotic normality of maximum likeli¬ 
hood estimates can be generalized to quasi-maximum likelihood estimates (estimates base<l 
on the assumption of a likelihood function which need not be the true function^. 

4. The Asymptotic Distribution of the Range. E. J. Gumbel, Newark Gollege 
of Engineering. 

The asymptotic distribution of the range w for initial unlimited distributions of the 
exponential type is obtained by convolution of the asymptotic distributions of the two 
extremes. Let a and w be the parameters of the distributions of the extremes for a sym¬ 
metrical variate, and let R = a(w — 2u) be the reduced range. Then the probability 
^(/2) of the reduced range is subject to the differential equation^" -j- 4^' — >1' exp (—R) «= 0 
which may be transformed into Bessel’s equation of the first order by the substitutions 
R — 2(log 2 — log z), and 'k = zU The solution is ^(/?) = zKiiz) for the asymptotic prob¬ 
ability, and ^(i?) = (z^/2)Ko(z) for the asymptotic distribution, Ko(z) and Ki(z) being the 
modified Bessel function of the second kind of orders zero and unity. Thus tables of '^(R) 
and }/^(R) may be calculated for any symmetrical distribuion of the exponential type. 
The distribution of the range iv for normal samples of size 10 is already very close to the 
asymptotic distribution provided that the parameters a and u are determined from the 
mean and the standard deviation of the range. This method permits the calculation of 
the distribution of the range for normal samples of any size larger than 10. 

5. The Comer Test for Association. John W. Tukey, Princeton University, 
and Paul S. Olmstfad, Bell Telephone Laboratories. 

Construction. In a scatter diagram, draw the two medians, that is, the median of the 
X values without regard to the values of y, and the median of the y values without regard 
to the values of x. Think of the four quadrants thus formed as being labelled -f, —, -f, — 
in order, so that the two positive quadrants lie along one diagonal and the two negative 
along the other. Beginning at the right-hand side of the diagram, count in along the ob¬ 
servations until forced to cross the horizontal median. Write down the number of ob¬ 
servations met before this crossing, attaching the sign, +, if they lay in the -|- quadrant, 
and the sign, if they lay in the — quadrant. Repeat this process, moving up from 
below’, moving to the right from the left, and moving down from above. The quantity to 
be used in the test is the algebraic sum of the four numbers thus written down. 

Distribution. The exact distribution of this quantity when no association is present 
and no two x'b and no two y’s arc alike is almost independent of sample size over the range 
of values where it is apt to be used. For example, a sum of 9 or more is exp)ected less than 
one time in ten for all samples of size 6 or more; a sum of 16 or more, less than one time in 
100 for all samples of size 10 or more; and a sum of 21 or more, less than one time in 1000 
for all samples of size 14 or more. Even for infinite sample size, the sums for these fractions 
become only 9, 14, and 19, respectively. 

Extensions. The same ideas that underlie the outside corner test for two variables 
may be extended in several ways to give tests for various types of association among three 
or more variables. 

6. Consistent Estimates Based on Partially Consistent Observations^ with 

Particular Reference to Structural Relations. J. Neyman and Elizabeth 
L. Scott, University of California. 
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Let (Xn) be a sequence of independent random variables and let Fx denote the distribu¬ 
tion of Xx . Each distribution Fx is assumed to depend on unknown parameters. If a 
parameter 9 appears in an infinity of distributions Fi, it is called structural. Otherwise, 
it is incidental. The sequence fXi,) is called consistent if has no incidental parameters. 
fXn) is called partially consistent if |F«) has both structural and incidental parameters.— 
Problem of fitting a straight line when both variables are subject to errors is that of a 
partially consistent series of observations. Let ^ and 17 or 4* be two linearly connected 
quantities, perhaps related to particular stars, where a and fi are unknown. The values 
and rjx corresponding to the ith star, (i -> 1 , 2 , • • * , s), are unknown. The observations 
provide measurements Xxi of ^*, (j - 1 , 2 , , m,), and measurements y**, (k — 

1 , 2 , • • • , n»), of Tlx . Both nix and n« are bounded and small. On the other hand, s may be 
considered as increasing without limit.—Assume that the x,, and the are normally 
distributed with variances o{ and <r\ and means and 17 , respectively. Then the totality 
of observations will form a partially consistent system with the structural parameters a, /3, 
a and d and with as incidental parameters.—If the observable random variables are only 
partially consistent, then the maximum likelihood estimates of the structural parameters 
(a) need not be consistent, (b) even if they are consistent and asymptotically normal, 
alternative estimates may exist which have the same properties but smaller asymptotic 
variances.—Consistent estimates of structural parameters may be obtained from modi¬ 
fied’* equations of maximum likelihood. The lower bound of the variance of estimates of 
structural parameters, provided by the Cram^r-Rao inequality, is attained only on certain 
conditions which are both necessary and sufficient. 



NEWS AND NOTICES 

Readers are invited to avbmit to the Secretary of the Institute news items of interest 

Personal Items 

Dr. Paul H. Anderson has been appointed Economic Analyst with the Market¬ 
ing Division, Office of Domestic Commerce, Department of Commerce, Wash¬ 
ington. 

Dr. Gilbert W. Reebe is now with the Division of Medical Sciences, National 
Research Council, Washington. 

Professor Harald C'ram^r, Director of the Institute of Mathematical Statistics 
of the University of Stockholm, was awarded the degree of Doctor of Science, 
honoris causa, by Princeton University on February 22,1947. Professor ('ram4r 
has acted as Visiting Professor of Mathematics at Princeton University and 
Yale University during the academic year 1946-^47. He will be at the Univer¬ 
sity of California at Berkeley during the 1947 Summer Session. 

Dr. Paul M. Densen has accepted a position with the Division of Medical 
Research Statistics, Bureau of Medicine and Surgery, Veterans Administra¬ 
tion, Washington. 

Mr. M. V. Divatia is now in charge of the office of the Statistician and Eco¬ 
nomic Adviser and Under-Secretary to the Government of Sind, Karachi, 
India. 

Mr. Clarence B. Fine, fonnerly with the Office of Price Administration, has 
transferred to the Bureau of Old-Age and Survivors Insurance, Social Security 
Administration, where he is employed as a Sampling Expert. 

Prof. Charles C, Grove was appointed Visiting Lecturer in Mathematics at 
the University of Pennsylvania for the sprmg semester. 

Assoc. Prof. E. E. Haskins of Northeastern University has been appointed to 
an assistant professorship at the Army Air Forces Institute of Technology, 
Wright Field, Dayton, Ohio. 

Prof. Roger Lessard of the Hull Technical School has accepted a position at 
the Ecole Polytechnique, Montreal. 

Mr. Edward D. Lowery is now a member of the Research Department^ Win¬ 
chester Arms Company, New Haven, Connecticut. 

Professor H. B. Mann of Ohio State University has been awarded the Frank 
Nelson Cole prize in the Theory of Numbers for 1946. 

Dr. Margaret P. Martin has been appointed to an assistant professorship in 
the Department of Preventive Medicine and Public Health, Vanderbilt Uni¬ 
versity Medical School, Nashville, Tennessee. 

Dr. A. L. OToole is at present employed by the Veterans Administration in 
the Washington headquarters, as Acting Chief of the Administrative Analysis 
Division in the Research Service. Dr. OToole was released from the Navy on 
September 23,1946, to inactive duty in the U. S. Naval Reserve, with the rank 
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of Commander. Dr. O’Toole served for nearly four yeai-s in the Xavy, in 
important administrative and statistical work for the Commander South Pacific 
Area and South Pacific Force. He will be remembered as having been with 
Admiral Halsey’s Pacific Fleet, and was awarded the 13ronze Star Medal. At 
the time of his release, he was Chief Staff Officer for Commander South Pacific 
Area and South Pacific Force. 

Mr. I. B. Perrott, since his demobilization from the British Aimy, has been 
Lecturer in Mathematics at the College of Technology and Commerce, Leicester, 
England. 

Mr. J. S. Ripandelli is now with the Actuarial Department of the Jefferson 
Standard Life Insurance C^oinpany of Greensboro, North Carolina. 

Dr. Ronald W. Shephard of the Univei*sity of California has been appointed 
to the staff of the Department of Mathematics, New York University. 

Mr. John R. Stehn is now a member of the Research liaboratory of the G^- 
eral Electric C’ompany, Schenectady, New York. 

Dr. Charles W. Vickery, formerly of Ohio State University, is engaged in work 
as a Research (Consultant in New York City. 


Miss Margaret Jeannin Dix, of the Univei*sity of CCalifornia Statistical Labora¬ 
tory, died an accidental death at her home in Berkeley on June 20, 1946. 

Mr. Albert M. Freeman, of the Boston Fiduciary and Research Association, 
died May 20, 1946. 

Dr. Walter Schilling, of the Stanford University Hospital, died suddenly in 
San Francisco, December 16, 1946. 


Summer Statistical Session at the University of California at Berkeley 

The important advances in the theory of statisti(!s during the war and espe¬ 
cially the unprecedented growth in the fields of application have created a 
strong demand for trained statisticians to till both the research and the teaching 
positions all over the country. Since in many cases the war time education had 
to be somewhat sketchy, unsystematic, and not very conducive to a thorough 
coverage of the vast material, it is felt that a relatively brief set of courses on a 
rather advanced level would be beneficial to many persons, both those who al¬ 
ready hold research or teaching positions in statistics, as \\'ell as those who 
prepare for higher degrees. 

With this object in mihd, the University of C’alifornia at Berkeley is offering 
a set of statistical courses during the Summer Session, June 23rd to August 2nd, 
1947. There will be three courses: (i) General Theory of Random Variables and 
Fmquency Distributions, by Harald Cram6r of the University of Stockholm; 
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(ii) Problems of Testing Hypotheses and of Estimation, by J. Neyman, Univer¬ 
sity of California, Berkeley; and (iii) Seminar Course. The last will be given by 
seven scholars, each giving tn o hours of lectures, as follows: 

1 . Statistical Astronomy. 

2 Orthogonal Polynomials and Problems of Moments. 

3. Methods of Calculation. 

(a) Gibbs* Methods in Statistical Mechanics 

j (b) Darwin-Fowler Method of Statistics. 

^ Large Scale Sampling Surveys. 

5. Statistical Problems Arising in Nuclear Physics 
Measurements 

6 . Problems of Population Genetics. 

7. Interactions between Industrial Problems and Mathematical 
Statistics. 

The purpose of the Seminar Course is to introduce the students either to 
branches of pure mathematics contingent on mathematical statistics but not 
ordinarily taught in the universities or to various fields of knowledge offering 
fruitful fields for statistical studies. 


Summer Statistical Session at Virginia Polytechnic Institute 

A Summer Statistical Session will be held at Virginia Polytechnic Institute, 
Blacksburg, Virginia, August 5 to September 5, 1947. This Session will be 
sponsored jointly by Virginia Polytechnic Institute, University of North Caro¬ 
lina, University of Michigan, Iowa State (^ollege, and the Federal Bureau of 
Agricultural Economics. 

The faculty ^vill consist of; Walter A. Hendricks, B.A.E., U.S.D.A.; Renis 
Likert, University of Michigan; H. L. Lucas, University of North Carolina; 
Maurice G. Kendall, England; George W. Snedecor, Iowa State College; Frank 
Yates, Rothamsted p]xperiment Station, England; Earl E. Houseman, B.A.E., 
U.S.D.A.; Raymond J. Jessen, Iowa State College, and Boyd Harshbarger, 
Virginia Polytechnic Institute. 

The following courses will be offered for credit: Engineering Statistics; Sta¬ 
tistical Methods; Design of Animal Experiments; Schedule Design and Interview 
Techniques for Sample Surveys; Sampling Design and Analysis; Mathematical 
Theory of Sampling; Seminar; Mathematical Statistics, and Experimental 
Design. 

In addition to the faculty, probable Seminar speakers are: W. F. Callendar, 
W. Q. Cochran, Miss Gertrude M. Cox, W. E. Deming, George Gallup, M. H. 
Hansen, Harold Hotelling, Arnold King, and Charles P'. Sarle. 

Inquiries regarding the Summer Session should be addressed to Boyd Harsh¬ 
barger, Professor of Statistics, Summer Statistical Session, Virginia Polytechnic 
Institute, Blacksburg, Virginia. 


R. J. Trumpler 
G. Szeg5 
V. F. Lrnzen 


P. C. Mahalanobis 

R. Berber 

S. Emerson 
H. ScHEFFi 
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New Members 

The following persons have been elected to membership in the Institute 
{January 1 to February BS, 1947): 

Asofsky, Samuel, B.S. (C.C.N.Y.) Stat., National Jewish Welfare Board, 1B66 E. IS St,^ 
Brooklyn SO, N. Y. 

Auer, Richard M., A.M. (Columbia) Instr. in Math., State Teachers Coll., Montclair, 
N. J., 88 No. 16 St., East Orange 

Bakan, David, M.A. (Indiana) Chief Stat., Comm, on Selection and Training of Aircraft 
Pilots, National Research Council, BS9 Natatorium, Ohio State Univ., Columbus 10, 
Ohio 

Beatty, Glenn H., A.B. (Ohio State) Grad, student and Fellow, Iowa State College, 

A, General Delivery, Ames, Iowa 

Campbell, Wallace A., B.S. (Columbia) Stat .Analyst, War Assets Administration, 4^S 
Washington Ave., Brooklyn 16, N. Y. 

Celia, Francis R., M.A. (Kentucky) Assoc. Prof, of Statistics and Director, Bur. of Busi¬ 
ness Research, Univ. of Oklahoma, Norman, Okla. 

Chapman, Douglas G., M.A. (Toronto) Asst. Prof, of Math., Univ. of British Columbia, 
Vancouver, Canada 

Cheydleur, Benjamin F., B.A. (Wisconsin) Chief, Mechanized Analysis, Naval Ordnance 
Lab., 60B Avenue E, District Heights, Washington 19, D. C, 

Coombs, Clyde H., Ph.D. (Chicago) AssH Prof, of Psychology, and Research Psychologist, 
Institute for Human Adjustment, Univ. of Michigan, Ann Arbor, Mich., 10B7 E. 
Huron 

Corton, Edward L«, Jr., M.B.A. (Chicago) Grad, student, Iowa State Coll., SOS Hodge 
Ave., Ames, Iowa 

Davis, Harold., A.B. (Brooklyn Coll.) Stat., Navy Dept., 416—SS St., S.E., Washington, 
D. C. 

Dutton, Arthur M., B.S.E.E. (Iowa State) Grad. Fellow, Mathematics Dept., Iowa State 
Coll., Ames, Iowa 

Fay, Edward A., A.M. (Harvard) Grad, student, Univ. of California, Berkeley, 41^ South 
17th St., Apt. BB, Richmond, Calif. 

Flanagan, John C., Ph.D, (Harvard) Prof, of Psychology, Univ. of Pittsburgh, Pitts¬ 
burgh 13, Pa. 

Gardner, Eric F., Ed.M. (Boston Teachers) Teaching Fellow and Milton Fellow, Grad. 
School of Educ., Harvard Univ., Cambridge, Mass., Walker House, 40 Quincy St. 

Gerende, Lincoln J., C.Ph.M,, U. S. Navy, Naval Medical Res. Institute, National Naval 
Medical Center, Bethesda 14, Md. 

Grossman, Evelyn, M.A. (Columbia) Stat., U. S. Dept, of Agriculture, 6401—14 St., 
N. W., Washington IB, D. C. 

Hill, Edwin A., Jr., M.A. (Columbia) Instr. in Math., Coll, of the City of N. Y., 60 West 
67 St., New York BS, N. Y. 

Horton, H. Burke, M.B.A. (Texas) Senior Transport Analyst, B90e Naylor Rd., S. E,, 
Washington BO, D. C. 

Horvltz, Daniel G., B.S. (Mass. State) Grad, student, Iowa State Coll., B1S7 Country Club 
Blvd., Ames, Iowa 

Ikhtiar-ul-Mulk, S. M., M.A. (Punjab, India) Grad, student, Princeton Univ., Graduate 
College, Princeton, N. 

Jaeger, Carol M., B.A. (Dubuque) Statistician, ISOO Columbia Terrace, Peoria 5, III. 

Jessen, Raymond J., Ph.D. (Iowa State) Bes. Assoc. Prof., Iowa State College, and 
Agric. Statistician, U.S.D.A., Statistical Lab., Iowa State Coll., Ames, Iowa 

Kinzer, Mrs. Lydia Greene, M.A. (Kansas) Ass't Instr. in'Math., Obdo State Univ., 
686 East Town Street, Columbus 16, Ohio 
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Langenhop, Carl £., M.S. (Iowa State) Instr. in Math., Iowa State Coll., Apt, 5, Cranford 
Annex, Ames, Iowa 

Lowy, Melltta E., A.B. (Hunter) Statistician, Grad, student, Columbia Univ., 645 Weet 
End Ave., New York B5, N. Y. 

MattHa, Sakari, Fil.Mag. (Helsinki) High School of Commerce, Helsinki, Finland 

Mayerson» Allen L*, l&.S. (Michigan) Grad, student and Teaching Fellow, Univ. of Mich., 
ISOB Packard St., Ann Arbor, Mich. 

McCreary, Garnet B., M.A. (Queen’s Univ.) Research Fellow, Statistical Lab., Iowa 
State Coll., Ames, Iowa 

McMillan, Olan T., M.A. (Michigan) Instr. in Math., Michigan State Coll., East Lansing, 
Mich. 

Morris, Edward B., A.B. (Indiana) Statistician, U. S. Bur. of Labor Statistics, 1915 Ridge 
Place S. E., Washington BO, D, C. 

Moshman, Jack, B.A. (New York) Tutor in Math., Queens Coll., Flushing, N. Y., 1B5~09 
Liberty Ave., Richmond Hill 19 

Natrella, Mrs. Mary G., B.A. (Pennsylvania) Statistician, Bureau of Ships, Navy Dept., 
IBlO—lBth St., N. W. Washington 5, D. C. 

Neal, T. Ellison, A.B. (Geo. Washington) Statistician, Textile Dev. Dept., U. S. Rubber 
Co., Hogansville, Ga. 

Noble, Carl E., Ph.D. (Iowa) Quality Methods Engineer, Kimberly Clark Corp., lAke- 
view Mill, Neenah, Wis. 

Ostle, Bernard, M.A. (British Columbia) Teaching Ass’t, School of Bus. Adm., Univ. of 
Minnesota, Minneapolis, Minn. 

Oztoby, Toby E., B.A. (Iowa) Grad. Ass’t, Dept, of Psychology, State Univ. of Iowa, 
Iowa City, Iowa 

Pelsakoff, Melvin P., Student, Princeton Univ., $4 North West College, Princeton, N. J, 

Rothschild, Colette, (Ecole Normale Superieure) Attachee de Recherches au Centre Na< 
tional de la Recherche Scientifique, 45 rue Madame, Paris VI*, France 

Slonim, Morris J., M.B.A. (Harvard) Statistician, Bureau of Labor Statistics, BIO Wayne 
Place S. E., Washington BO, D. C. 

Soler, Reuben I., B.B.A. (C.C.N.Y.) Statistician, Food and Drug Administration, B46 
Portland St., S. E., Washington, D. C. 

Stouffer, Samuel A., Ph.D. (Chicago) Prof, of Sociology and Director of the Laboratory 
of Social Relations, Emerson Hall, Harvard Univ., Cambridge, Mass. 

Teicher, Henry, B.A. (Iowa) Graduate student, Columbia Univ., 1S9 Osborne Terrace, 
Newark, N. J. 

Tiedeman, David V., M.A. (Rochester) Instr. in Educ., Grad. School of Educ., Harvard 
Univ., Walker House, Quincy St., Cambridge $8, Mass. 

Tintner, Gerhard, Ph.D. (Vienna) Prof, of Economics and Mathematics, Iowa State 
Coll., Ames, Iowa 

Weiss, Eleanor S., Ed.M. (Boston Teachers) Teaching Fellow, Grad. School of Educ., 
Harvard Univ., B006 Commonwealth Ave., Brighton S5, Mass. 

Wilson, William A., Jr., A.B. (California) Teaching Ass’t in Psychology, Univ. of Calif., 
l^rkeley 4, Calif. 

Woodell, Allan D., A.B. (N. Y. State Teachers, Albany) Graduate student in math., Univ. 
of Mich., 455 Church St., Ann Arbor, Mich. 


Omitted from 1946 lists of new members: 

Feraud, Prof. Lucien, Faculte des Sciences Economiques et Sociales, Univ. de Geneve, 
B4 rue Henri Mussard, Oenbve, Switzerland 



REPORT ON THE ATLANTIC CITY MEETING OF THE INSTITUTE 

The Ninth Annual Meeting of the Institute of Mathematical Statistics was 
held at Atlantic City, New Jersey, on Friday and Saturday, January 24-25,1947. 
The meeting was held in conjunction with meetings of the American Economic 
Association, American Statistical Association, and the Econometric Society. 
The following 154 members of the Institute attended the meeting: 

Beatrice Aitchison, F. L. Alt, R. L. Anderson, T. W. Anderson, K. J. Arrow, Max Astra- 
chan, B. M. Bennett, Joseph Berkson, A. J. Berman, C. I. Bliss, Paul Boschan, A. E. 
Brandt, M. F. Bresnahan, Philip Brown, 0. P. Bruno, R. W. Burgess, 0. K. Buros, B. H. 
Camp, F. R. Celia, Uttam Chand, K. L. Chung, C. W. Churchman, P. C. Clifford, W. J. 
Cobb, W. G. Cochran, F. G. Cornell, D. R. Cowan, Harald Cram6r, J. H. Curtiss, J. F. Daly, 
G. B. Dantzig, D. G. Deihl, D. B. DeLury, B. W. Dempsey, H. F. Dorn, F. W. Dresch, 
A. J. Duncan, David Durand, P. S. Dwyer, Churchill Eisenhart, W. D. Evans, Will Feller, 

C. D. Ferris, Irving Fisher, L. R. Frankel, M. A. Geisler, Leon Gilford, M. A. Girshick, 

C. H. Graves, K. E. Greene, S. W. Greenhouse, F. E. Grubbs, E. T. Gumbel, Margaret 
Gurney, Louis Guttman, Trygve Haavelmo, K. W. Halbert, M. H. Hansen, Miriam S. 
Harold, T. E. Harris, Boyd Harshbarger, Bernard Hecht, Wassily Hoeffding, H. B. Horton, 
Harold Hotelling, E. E. Houseman, Helen M. Humes, Leonid Hurwicz, Seymour Jablon, 
R. W. James, R. J. lessen, H. L. Jones, Alice S. Kaitz, H. B. Kaitz, L. S. Kellogg, H. S. 
Konijn, Tjalling Koopmans, C. F. Kossack, R. L. Kozelka, D. H. Leavens, Howard Levene, 
J. E. Lieberman, Rensis Likert, S. B. Littauer, Irving Lorge, P. J. McCarthy, P. W. Mc- 
Gann, F, E. McIntyre, H. F. MacNeish, J. D. Maddrill, Jacob Marschak, Max Millikan, 

A. M. Mood, Mrs. Margaret Moore, J. W. Morse, J. E. Morton, Frederick Mosteller, D. N. 
Nanda, P. M. Neurath, Jerzy Neyman, M. L. Norden, Nilan Norris, H. W. Norton, P. S. 
Olmstead, E. G. Olds, Sophie Rakesky, Chester Rapkin, Olav Reiersol, W. A. Reynolds, 
P. R. Rider, C. F. Roos, A. C. Rosander, Ernest Rubin, Herman Rubin, P. J. Rulon, Frank 
Saidel, Marion M. Sandomire, Max Sasuly, F. E. Satterthwaite, E. D. Schell, E. M. Schrock, 

D. H. Schwartz, G. R. Seth, L. W. Shaw, W. A. Shewhart, J. II. Smith, R. T. Smith, Leslie 

E. Simon, Milton Sobel, C. M. Stein, G. T. Steinberg, Joseph Steinberg, H. W. Steinhaus, 

F. F. Stephan, A. P. Stergion, M. S, Stevens, G. J. Stigler, S. A. Stouffer, Zenon Szatrowski, 

B. J. Topping, J. W. Tukey, D. F. Votaw, Jr., rfelen M. Walker, J. H. Watkins, Louis 
Weiner, Samuel Weiss, S. S. Wilks, Elizabeth W. Wilson, C. P. Winsor, J. Wolfowitz, M. A, 
Woodbury, Holbrook Working, C. A. Weight, and T. O. Yntema. 

The first session, a joint session with the Econometric Society and the Bio¬ 
metrics Section of the American Statistical Association, was held at two o’clock 
on Friday afternoon, and was devoted to the topic, Applications of Statistical 
Techniques to Agricultural Economics, Holbrook Working of Stanford Uni¬ 
versity presided. The following four papers were presented: 

1 . Use of Variance Components in the Analysis of Market Differentials in Hog Prices. 
R. L. Anderson, University of North Carolina. 

2 . An Application of the Analysis of Variance in the Economic Evaluation of Production. 
Boyd Harshbarger, Virginia Polytechnic Institute. 

3. A Model of the Economic Interdependence belv)een Agriculture and the National Economy. 
Trygve Haavelmo, Cowles Commission for Research in Economics. 

4. The Reduced-Form Method for Estimating Simultaneous Economic Relationships. 
M. A. Girschick, Bureau of the Census. 
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The session concluded with a discussion of these papers by T. W. Anderson, 
Columbia University; Milton Friedman, University of Chicago; and, Harold 
Hotelling, University of North Carolina. 

At 8 o’clock on Friday evening there was a joint session with the Econometric 
Society and the American Statistical Association, on the topic. When is Ike 
Analysis of Variance Useful in Economic Research? Arthur R. Tebbutt of 
Northwestern University presided, and the following three papers were presented: 

1. The Advantages of the Analysis of Variance for Research and Managerial Control 
Purposes, Harry Pelle Hartkemeier, University of Missouri. 

2. Estimation of Economic Relationships and Multivariate Regression. 

Leonid Hurwicz, Iowa State College. 

3. Nonstandard Forms of Variance Analysis. 

W. Allen Wallis, University of Chicago. 

There was discussion of these papers by Tjalling Koopmans, Cowles Commission 
for Research in Economics: Gerhard Tintncr, Iowa State College; and, J. W. 
Tukey, Princeton University. 

At 10 o’clock on Saturday morning there was a joint session with the American 
Statistical Association devoted to the topic. Use of Ordered Observations in 
Statistical Analysis^ with Harold flotclling of the University of North Carolina 
as chairman. The following two papers were presented: 

1. Estimation of Parameters by Use of Order Statistics. 

Frederick Mosteller, Harvard University. 

2. Tolerance Limits. 

Jacob Wolfowitz, Columbia University. 

There was discussion of these papers by John H. Smith, Bureau of Labor Sta¬ 
tistics; Howard L. Jones, Illinois Bell Telephone Company; and J. W. Tukey, 
Princeton University. 

At the Saturday morning session one contributed paper of the Institute of 
Mathematical Statistics was also presented, by E. J. Gumbel, Newark College 
of Engineering, on the topic: The Asymptotic Distribution of the Range. 

The Institute’s session at 2 o’clock Saturday afternoon was devoted to con¬ 
tributed papers. W. G. Cochran, president of the Institute, presided, and the 
following four paper’s were presented: 

1. A Test of Significance of the Coefficient of Rank Correlation for More than Thirty Ranked 
Items. 

Nilan Norris, Hunter College. 

2. A Generalized T Measure of Multivariate Dispersion. 

Harold Hotelling, University of North Carolina. 

3. Asymptotic Properties of Maximum and Quasi-Maximum Likelihood Estimates. 
Herman Rubin, Cowles Commission for Research in Economics. 

4. 7'he Corner Test for Association. 

J. W. Tukey, Princeton University, and Paul Olmstead, Bell Telephone Laboratories. 
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Abstracts of these papers appear elsewhere in this issue. 

Following the session on contributed papers, Professor Jerzy Neyman of the 
University of California gave an invited address on the topic: On Consistent 
Estimates^ with Particular Reference to Structural Relations between Several Vari¬ 
ables all Subject to Random Error. A discussion of this address followed, by 
Miss E. L. Scott, University of California; A. Wald, Columbia University; and 
Tjalling Koopmans, Cowles Commission for Research in Economics. 

The meeting closed with the annual business meetmg of the Institute, which 
was held at 5 p.m. on Saturday in Haddon Hall. Reports by the President, 
Secretary-Treasurer, and Editor were followed by the election of officers for 
1947: Will Feller, President; Morris H. Hansen and John H. Curtiss, Vice- 
Presidents; and Paul S. Dwyer, Secretary-Treasurer. 

P. S. Dwyer, 
Secretary. 
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Introduction. If n real variables , a:-2, • • • , are subject to a probability 
distribution with the element dVi{x\)dV2{j^2) * • • dVn{xn) one can ask for the 
distribution of any function / of Xi, , • • • Xn . We are primarily interested in 

siatishcal junctions^ i.e. in functions that depend on the repartition Sn{x) of the 
n quantities , 0*2, • • • a;n only. The simplest case is that of the linear statis¬ 
tical functions 

(1) / = f dSnU) = - [^(a-,) + +■■■ + 4'MI 

J n 

The so-called Central Limit Theorem of Probability Calculus states that the 
distribution of a linear statistical function, if n tends to infinity, approaches 
more and more the normal (Gauss) distribution if some very general conditions 
linking ^(a;) and the Vp{x) are fulfilled. It has been shown, ten years ago, [2] 
that the restriction to linear functions here is immaterial. Much more general 


1 The function Sn{x) is called the repartition of the real quantities xi, x*, • • • , aj„ if 
nSn(x) IS the number of those among the xi , X 2 , • • •, x» that are smaller than or equal to x. 
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statistical functions tend towards normalcy with increasing n, for example the 
variance of mth order 

(2) f=M„ = f(x- ardSnix), a = f xdS„(x) 

and, likewise, such combinations as the Lexis quotient M^/a{y. — ajN) or Gini’s 

disparity measure 1 — J (1 — SnYdx/a or, in the multidimensional case, the 

correlation coefficient, etc. On the other hand, statistical functions are known 
whose distributions assume, as 3 miptotically, a form different from the Gaussian. 
One example is Pearson's Chi-square, another the test function introduced 
by H. Cramer [ 1 ] and the author [4]: 

(3) / = „^ = I g'{x){S.{x) - ?„(*)]* dx 
where g\x) > 0 and 

(4) r.W = 1 [FiW + y,(x) + • • • + FnW). 

n 

N. V. Smirnoff [7, 8 ] computed the asymptotic distribution of o? for the case 
that all Vp{x) and, therefore, Vnix) equal one and the same distribution func¬ 
tion V(x). The result differs widely from the Gaussian distribution. 

In order to understand all this it is necessary to consider / as a function de¬ 
fined in the space of distributions V{x) (or in a sub-space of it). Then, the vari¬ 
able / whose distribution is sought is the value of f{V{x) } at the “point" Sn{x) 
and should be written as/{<S«(a;)). Such “functions of functions" were first 
introduced by Vito Volterra (1887) and are today a familiar topic of higher 
analysis. The first statement that can be made is that the asymptotic dis¬ 
tribution of /{/Sn(:r)} depends mainly on the behavior of f{V(x)] at the point 
Vnix) defin^ by (4). 

Volterra also introduced the notion of derivatives and of Taylor development 
for a “fonction de ligne." Using these concepts a more specific statement can 
be pronounced: The type of asymptotic distribution of a differentiable statistical 
function /{> 8 n(a;)) depends on which is the first non-vanishing term in the Taylor 
development of f{V(x )) at the point Vn{x); if it is the linear term the limiting dis¬ 
tribution is normal, under restrictions that can easily be derived from the Central 
Limit Theorem-, in other cases higher types of asymptotic distributions result 
The present paper tries to establish this theorem and to furnish preliminary 
information about the as 3 rmptotic distribution of the second type. 

If both the function/{^(a;)} and the sequence of distributions Vi{x), V 2 {x), 
1 ^ 3 ( 0 ;), • • • are defined independently of each other, it cannot be presumed that 
the derivative of f vanishes at t^n(a;). In this sense the normal distribution ap¬ 
pears as the “general case" of an asymptotic distribution while the higher types 
represent certain “singularities." In the case of type m, (m ~ 1, 2, 3, • • •)> 
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the distribution of the expression 

(5) n"''Vmx)} - f{?.(x)]] 

tends towards a function of bounded mean value and variance. For m = 1 
it is a Gauss function with mean value 0 and finite variance. For any uneven 
m the distribution is symmetrical with respect to the zero point. If / is given, 
the limiting distribution is essentially determined if in addition to Vnix) one /unc- 
tion of two variables, Unix, y), is known. 


( 6 ) 


Unix, y) = - i: [v,ix) - V,ix)V,iy)\, 
n 

= - E [VM - V,ix)V,iy)], 


ix g y) 


(x ^ y). 


For instance, in the case of the linear function (m = 1) defined in eq. (1), the 
(second order) variance of (5) is found as the Stieltjes integral 


(7) 


f Hx)^iy) dUnix, y) 


and no mean values of higher order are required for computing the moments of 
any order, whatever m is. 

For m = 2 the complete expression for the characteristic function of the asymp¬ 
totic distribution of (5) is developed in Part III of this paper. It has the form 

DM 

where Z)(X) is in general the Fredholm determinant of a symmetrical kernel that 
depends on the second derivative of f{Vix) \ at F = Fn , on Vn and on C7„ . 
If the Vvix) are discontinuous distributions with saltus at k distinct points only, 
D is the determinant of a quadratic form of k variables. This happens to be 
the case with Pearson’s while the distribution found by Smirnoff represents 
a fairly general case of the asymptotic distribution of second type. 

PART I. PRELIMINARY THEOREMS 

1. Asymptotically equal distributions. Let Ki, K^, Ki, ••• be an infinite 
sequence of collectives, kn the number of variables in Kn and An , Bn two func¬ 
tions of these variables, (n = 1, 2, 3, • • •)• The cumulative distribution func¬ 
tions of An and Bn will be denoted by Pnix) and Qn{x) respectively, i.e. 

(1) Pnix) = Prob {An ^ x], Qnix) == Prob [Bn ^ x] 
and the expectation of j A„ — Rn 1 by 

(2) En{ 1 An - Rn 1 } 

all these quantities being taken with respect to the distribution in Kn . 
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Two functions Fn{x) and Gn{x) both depending on the parameter n are said 
to be asymptotically equal if 

(3) lim I Fn(x) — Gn(x) | = 0 uniformly in x. 

nmtOO 

If this is the case for the cumulative distribution functions Pn{x) and Qn{x) of 
An and Bn we shall also say that An and Bn have the same asymptotic distribu¬ 
tion. Eq. (3) will also be written as Fnix) ^ Gn{x). The following can be 
proved: 

Lemma A. If vnth increasing n the expectation of the absolute difference be¬ 
tween An and Bn tends towards zero and if one of the functions Pn{x) or Qn{x) is 
asymptotically equal to a function Fnix) that has a uniformly bounded derivative, 

i.e. 

(4) lim |) = 0, < M for all n 

dx 

then An and Bn have the same asymptotic distribution. 

This statement, in a slightly different wording, was proved in an earlier paper 
[2] and the proof will not be repeated here. If one of the various definitions for 
“stochastical convergence” is used, one can also say that An and Bn , under the 
stated conditions, converge stochastically towards each other. 

The Lemma A can be extended and modified in various ways. First, it is 
obvious that the expectation of [ A« — Bn \ can be replaced by that of any 
positive power \ An — Bn\^- With respect to one could ask for the existence 
of a bounded derivative in all points except for a zero set only. Then Pn and 
Qn would still converge everywhere except for this zero set and the definition 
of asymptotically equal distributions could be extended to this case. In the 
present paper this will not be done as it is not our purpose to strive for results 
of the possibly greatest generality. 

2. Special class of statistical functions: quantics. Preliminary to the study 
of general statistical functions a special class which corresponds to quantics 
(homogeneous polynomials) of mth order must be discussed. Let FiCx), V 2 ix), 
V^ix ), • • • be the cumulative distribution functions in a sequence of one-dimen¬ 
sional collectives Ci, C 2 , Ca, • • • and Snix) the repartition of a sample drawn 
from the n-dimensional collective Kn , with the distribution element 

dViix{)dV2ix2) •••d7n(a:„). 


We introduce 


Tn(x) = ^„(X) - = - Z V,{X). 

n *<-.1 


( 5 ) 
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Here, nTn{x) is obviously the excess of observed values ^ x over their expected 
number. Quantics of first, second, third, • • • order are then defined as 

/ilSn(x)} = / mdT.ix) 

( 6 ) MSnix)} = // y)dTMdTM 

/3{Sn(x)l = /// y, z) dTn(x) dTniy) dTn{z) 


all integrals to be extended over the total range of x. Of course, only such ^ 
for which the respective integral exists are admitted. The first, /i, is obviously 
a linear statistical function and the asymptotic distribution of '\/nfi is, under 
well-known conditions, a Gauss function with the mean value zero and the 
variance given in eq. (7) of the Introduction. In / 2 , /s, • * ■ the ^ may be 
supposed to be symmetrical with respect to their variables. It will be seen 
later (Part II, sec. 2) that the first derivative of /o, the first and second deriva¬ 
tives of fi , etc. vanish at the point Vnix), 

All the above functions /i, / 2 , .fs, • * * can be considered (if the are continu¬ 
ous) as the limits of ordinary quantics in k variables. Choose k disjoint inter¬ 
vals J\ i h y • • • , //; on the a;-axis, and call /jt+i their complement. Denote the 
increment of 7v(^) within Ik by and the increment of Sn{x) by pn«. Ob¬ 
viously PvK is the probability, within Cy y oi x falling in the interval Ik and np^ 
is the number of observed sample values in the same interval. We introduce 
the excess values : 


PnK Pni 


1 ” 

pn. = - L Pyi 

n v -1 


and form the sums 

(S) /l ~ y fi ~ > /s ~ y * * * • 

Ktml X.X K.X.M 

By selecting suitable sets of intervals h y h , " * y h and appropriate values 
for the constants , ^kx , • • * , one can approximate the integrals ( 6 ) by sums 
of the form ( 8 ). 

Our next task will be to find asymptotic values for the expectation and for the 
moments of the quantities defined in ( 8 ). Clearly a formula for the expectation 
of a power product • • • where a, jS, 7 , • • • are positive integers, is the 

only thing we need. To arrive at such a formula we replace each of the one¬ 
dimensional collectives C, by a fc-dimensional C* in the following way. 

In C* the chance variable is a ^-dimensional vector which can take (A; + 1) 
distinct values only: it can be zero or coincide with the unit vector parallel to 
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one of the k axes. To the latter values of the variable we assign the probabilities 
Pfi , , * • * I Ppk and to the zero the probability 

(9) Pr.fc+l = 1 - PfI “ Pf 2 ~ - PfA: 

This quantity, of course, may vanish. The mean value of Ct is the point with 
the coordinates p^i, Pf 2 , • • • , Ppk . 

If the n collectives , C* , • • • , Ct are combined, the sum of the n observed 
vector values is a vector with the components npni , npn 2 , * * * , • If in 

each Ct the origin is shifted to the mean value and the coordinates with respect 
to the new origin are called Zi, 22 , • * * , , the sums of the observed Zi, 22 , • * * , 

2 *-values will be nji , nfa, * * * , rather than npni , np„ 2 , • • • , np„k . Thus 
it is seen that all questions concerning the distributions of f 1 , { 2 , fg, * • • can 
be answered on the basis of the well-known rules on the addition of n independent 
chance lariahles. This leads to the symbolic formula for the expectation: 

( 10 ) fin!• • • ! = (z (z , 

where on the right-hand side each term 

( 11 ) 

has to be replaced by 

(110 J ■■■ dV^z). 

Here, obviously, Vt{z) is the distribution function in C* and the expressions 
(11') are in fact sums of (A; + 1) terms, for example 

/ ZiZ2dV*(,z) = p,i(l - p.i)(-Prt) + Pv 2 (-p,i)(l - P^) 

( 12 ) 

+ Z p..(-p.l)(-p»s) = —pptPyl • 
1-8 

It will be seen in the next section that only very few of these sums are needed 
for computing the asymptotic value of (10). Note that the value of (11') can 
be expressed in terms of p,i, Pf 2 , Pfs , * * • alone if Ji, ^ 2 , & , • * * only appear in 
the product. 

3. Asymptotic expectation of excess-power products. We first consider the 
case where the sum of exponents a, jS, 7 , • • • is an even number 

(13) o£ -|“ /3 4“ T 4“ * * * ~ 2in, 

On the right-hand side of (10) stands a sum of n*”* terms, each a product of 2m 
factors Z„ . It follows from ( 11 ') that the absolute value of a product cannot 
surpass 1 . The second subscripts are the same in each term: first a ones, then 
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jS twos, 7 threes, etc. The first subscripts are in each term a combination of 
2m digits out of v = 1, 2, 3, • • • ^ n. The number of those combinations which 
include s different j^-values, (s = 1, 2, • • • 2m), is 

(14) . 0)[.“ - (;)(. - D- +... + 1-]. 

Obviously, the are bounded (independent of n). 

If 5 > m the combination of first subscripts must include at least one v-value 
that appears only once. All those products vanish since 

(15) J z^dV^iz^) = 0 for all k, p 


due to the fact that the origin in the «-space coincides with the mean value of 
the distribution Vt(z). Note that 


(16) 



(s < m) 
(8 = m). 


It follows that the sum of all terms in (10) that correspond to any s < m are 
of the order ©(n"”) or smaller. 

Thus, we arrive at an asymptotic expression for En by dividing both sides of 
(10) by tT: 

(17) En[il * \ ^ (II ^ i'k) 


where only such products on the right-hand side are retained which include 
exactly m different v-values each appearing twice. 

In analogy to (12) we compute 

/ Z^Z4Vy(z) = —PwtPvK (t 7*^ k) 

(18) 

“ Pi't(l 


and write, for the sake of abbreviation 

(19) p.':’ = - p..p« = p»’ 


with the usual meaning o{ 6 Oift k and = 1 if t = k). Then the siun 
to the right in (17) includes (2m!)/2”* terms, each a product of m factors 
If each of the m couples i, k consists of two different figures, the respective prod¬ 
uct appears a \ ply I • • • times; if r couples are doubles (t = k) the multiplicity 
of the term is 2”’‘al piy! • • • . Therefore, (17) takes the form 


( 20 ) 


- E ••• pi:t- 
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In this sum the upper indices are any set of m digits out of 1, 2, 3, • • • , n 
and the subscripts are all sets of m couples including a ones, 13 twos, y threes, 
etc. To each such set of m couples belong (Z) terms of the sum. The number 
of sets of couples is bounded (independent of n) . The exponent r is the number 
of doubles (t = k) among the m pairs. 

The expression (20) admits of a transformation which renders it much more 
suitable. Assume that a set of couples i, k has been chosen according to the 
conditions and consider the product 

(21) (E ■ • •)(r 

Among the n” terms which we obtain by developing (21) are all terms appearing 
in the sum (20), each of them repeated m \ times and, in addition, 

(22) n*” — (m)m! = n"" — n(n — l)(n — 2) • • • (n — m + 1) 

other products of m factors P. Since the difference (22) divided by goes to 
zero with increasing n and each | P | is smaller than 1, the additional terms 
have no importance. We therefore introduce the quantities 

(23) P.. = - E p\:^ = «..-E pn - - E . 

n i/.i n 


Then (20) can be written as 


(24) 


W! t.it 


Here we have a sum of a finite number of terms. It will be supposed in all that 
follows that the P,* as defined in (23) do not vanish identically as n increases in¬ 
definitely. 

Since in the sum (24) no upper indices appear, equal terms repeat themselves. 
We can, therefore, rearrange it, using the polynomial coefficients and absorbing 
at the same time the factor 2“'^. The final form of (24) is given in the following 
Lemma Bi, which also includes a statement for the case of an uneven sum of 
exponents a + + 7 + * * * • In fact, it is easily seen that if again half the 

sum is called m, no group of terms on the right-hand side of (10) exists that 
would supply a finite limit when divided by n"". Thus we arrive at 

Lemma Bi . // nj* is the numerical excess of observed over expected quantities 
falling in the interval /«, the asymptotic expectation of the excess-power product 
••• is given by 

(V^n)“^^^^ if <x + + y + • - • uneven 


(25) 


a !jS ! 7 ! 


Xmmtl I I 

ff <rillO’22i 


<ri 2 l 


(iPu)'"(iP 2 =)' 


P aia T5 9] 
12 riz 


a + jS + 7 + • * • even 
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the sum to he extended over all sets of non-negative integers tru, 0 - 22 , • * •, ani • • • 
ihatfulfiU the conditions 

(250 ^11 ~ <^12 — <rii — • • •)> 0'22 = — o'2i ““ (Tja — • • •)> * * * 

The Ptg as defined in (23) depend on two groups of mean values only^ namely on 

(25") = - 23 P« and p,p. = - 23 P« • 

n n v.i 

Some properties of the matrix will be discussed in the next Section. 

For practical computation, instead of (25), a recursion formula may be used 
which follows immediately from (24). Writing simply (a, jS, 7 , • • •) for the sum 
in (24) the formula reads 

(26) - 2, /3,t, • • OPii + K“, /3 - 2, 7 , • • •)?« + • • • 

+ (a - l,/3 - 1 , 7 , •••)P«+ (a,/9 - 1,7 - 1), ■••)?*.+ •••• 

If all the original distributions Vp{x) are equal, this recursion formula, and from 
it (25), can be derived almost immediately from the theorem on the multiplica¬ 
tion of characteristic functions with the addition of chance variables. 

Note that the expectation of the product is P^Jn for any value of n. 


4. Asymptotic expectation and variance of quantics. We first state a char¬ 
acteristic property of the expression (25) for the expectation of an excess power 
product. Let us denote by the right-hand side of (25) in the case 

of even a + + 7 + • • •. Then, if .. is expressed in terms of Pj* and each 

time the subscript 2 is changed into 1 , we arrive at the value of Ca+ 3 . 0 . 7 ..... 
This would not be the case if were expressed in terms of p ^, since e.g. 

Cii = Pii = Pi - pipi, C12 = P12 = ~PlP2 . 

In order to prove the statement we observe that the can be derived 

from the coefficients in the development of the mth power of a quadric: 


(27) 




=m\ yi 


C'g,9,7,..- 

a\p\yl • • • 




It follows that 


(270 




Jl 6 ^ 

••• 


tiiC 


If in the subscripts of P*, the ones and twos are identified, the quadric becomes 
a function of , < 4 , • • • and the derivative with respect to d ti dtS equals 

the derivative with respect to d ti'^^. On the other hand, the latter derivative 
corresponds to the value of Ca+/ 9 .o,r,- - in the form (27'). 

Taking m = 2 , a = 0 = 7 = 5 = 1 , eq. (25) supplies 

( 28 ) n^En{ftfK{x{/il P»«Pxm + PixP«M + PiltP^ • . 
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According to the above statement this is correct whether t, k, X, /n are or are not 
different from each other. Thus, if is a symmetric set of constants, we 
have 

(28') En[ 5^ 3 X) ^i«XMPiit?XM* 

In general, the numerical factor to the right, i.e. the number of sets of couples 
drawn from 2m figures, is ( 2 m)l/ 2 "*m! = 1*3 ( 2 m — 1). Thus we can 

state: 

Lemma B 2 . If a guantic Am is defined according to ( 8 ) with symmetric coeffi¬ 
cients, its asymptotic expectation is given by 

(29) ~ 1.3.5 • • • (2m - DE A .4 • • • • 

Before applying this to the continuous case defined in ( 6 ), let us consider some 
characteristic properties of the matrix P**. According to the definition (19) 
of we have 

(30) ‘e p|r>'. /«= Z p.. t\ - ^E Pv. 
and using (9) one easily derives from Schwarz’ inequality 

Since Pi« is the arithmetical mean of the it follows that the matrix P^* 
is at least semi-definite and is positive definite except when all = O’ 

In the latter case (if e.g. the k intervals cover the whole x-axis) one has 

(31) Z r.. = - Z Z p« - (Z p«) = - Z p»,t+i(i - p..iM-i) = 0 

i|K n Vmml [_ l.i>l / J ^ Vmml 

which shows that here the reciprocal matrix P* does not exist. 

In the “complete” case, that is, with all Pr,jfc+i = 0, the elements in each 
horizontal or vertical line of the matrix P*, have the sum zero. It follows that 
the k homogenous equations sP.,Xi = 0 have the solution Xi = a ^2 = • * * = Xh 
and, therefore, that the cofactors of all elements of Pt* have one and the same 
value. For each single v the determinant of can be computed: 

j PiK I ~ * • * PwkP»,k+l 

If this is applied to the principal minors of the same determinant in the case 
Pp,k+i = 0, one finds the characteristic equation of the matrix Pjj^ to be 

1- xp.':’ I ^ ~ 

This shows that (k — 1) characteristic roots separate the abscissas l/p,i , 
1 /Pp 2 , • • • , I/Pp* (one root being zero). 

The number k of intervals has nothing to do with the preceding argument 
leading to the eqs. (25) to (28). Also can the entire computation be repeated 
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in terms of dT^ix^y dTn(x^), dTn{xz ), • • • instead of fi, {2, • if appro¬ 

priate differentials are substituted for the . To find the latter ones we note 
that ppit stands for the increment dV^ix), Thus, using d{x, y) in analogy to 
(= 1 for x = 2/ and = 0 for x 5*^ 2/) we set 

(32) 

= 5(x, y) dV,{x) - dWp{x, y) 

which is equivalent to the definition of a function of 2 variables: 

( 33 ) ““ 2/) ^ y) 

= V.iy) - 7,(x)F,(t/) = 7 .( 2 /) - T7,(x, y) (x ^ 2 ^). 

Then has to be replaced by 

(34) dVn(x, 2/) = - S dU,(x, y) = 5(x,y) dVn{x) - dT7„(x, 2/). 

n i>.i 

This dUn{Xy y) is the expectation of dTn{x) dTn{y)/n. 

The function 


(35) 


Un(x,y) =l-t:U,(x,y) 


is the difference of two cumulative distribution functions, one corresponding to 
a distribution along the straight line x = 2/ with the element dVnix) and an¬ 
other distribution over the whole plane with the element 

(350 dTr„(x, y) =-t. dVM dV,(y). 

n V .1 

To each one-dimensional distribution 7„(x) belongs one ^^distribution excess** 
Up{Xy y) as defined in (33). The are the increments of Uy{x^ y) within 
the product interval dxdy. It is seen from the preceding argument that the 
asymptotic moments of any quantic (6) or (8) depend only on the average Vn 
of the distribution excesses . 

If a quantic is defined by (6) and the integrals on both sides exist, the asymp¬ 
totic expectation of /2m may be written in formal analogy to (29) as 

n^Enifim] 1.3.5 • • • (2m ^ ff ''' f > * * • > ^m) 

(3fi) 

X dTJniXi I X2 ) dUniXi f X4) • * • 1 

This formula is identical with (29) if ^ has constant values in a finite number 
of intervals and vanishes outside these intervals. But it will be seen in the next 
section that (36) can be used in more general cases also. 

For the sake of practical computation one may develop the righthand side 
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of (36) into terms explicitly depending on the given averages 'r»(x) and Trfi(a:, y). 
For example, in the case m = 3: 

w*En{/«} ^ 1.3.5 JJJ [ypixi yXiyX2,X2,Xi,X3) dVnixi) dfn(xa) dVnixz) 

- 3^(a;i, Xi , Xi , ar2, Xa, 0 : 4 ) dfrixi) dVn{x2) dWnix ^, x^) 

+ mxi , , 0 ^ 2 , a;a, 0:4, aJs) dVnixi) dWn{x2 , Xs) dWnix ^, Xa) 

- ^(xi, X2, X3, X4, X 6 , Xe) dTFnCxi, X2) dTFnCxa, X4) dWn(xt , Xa)] 

In the general case, the numerical factors in the m-tuple integral are the binomial 
coefficients of order m. 

The higher moments of quantics fm can be computed in the same way as 
Enifm} since any power of fm is a quantic again. The formulas, however, be¬ 
come more involved since the coefficients of fm are not immediately given in a 
symmetric form. It will suffice to show here how the (second order) variance 
of /z can be found. The second moment is the expectation of 

(39) fl = fffi ^(x, yMz, u) dW dTM dT„{z) dTn{u). 

Applying here eq, (28) we have 

n^En{fl} ~ jj yl/{x, y)\l/{z, u)[dt7„(x, y) dU„(z, u) 

(40) _ . - . 

+ dUn{x, z) dVniy, u) + dU„(x, u) dUn(y, «)]. 

The first term in the brackets leads to the square of n Bnlf^} while the second 
and third terms, due to the symmetry of ^(x, 1 /), supply two equal integrals. 
Thus 

Var {n/ 2 ) ^2 jj ^(x, y)}P(z, u) dUn{x, z) dUniy, u) « 

(41) 2 jJ ^(x, x)\l/{y, y) dVn{x) dVniy) ^ jJ ^(x, 2 /)^( 2 /, z) dVniy) dWn{x, z) 

+ jj ^{Xy y)}p{Zy u) dWn{xy z) dWn(:y, w) j- 

In the same way moments and variances of any order can be computed for any 
quantic fm . 

5. Final statement on the limit of expectation of quantics. We shall prove 
the following: 
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Lemma Bs . Given a sequence of distributions Vi{x)y V 2 {x), Fs(a;), • • • and a 
quantic of order 2m 

* * * f ^ > * * * > ^ 2 m) dTn{Xi) dTfSjX^ • • • dTft(jX2n^ 

assume that there exist a continuous function and a distribution ^(x) sueh tha^ 

, I ^(^1 , ^2 , • • • X2nd I ^ 

dV,{x) g dV(,x) for | a; | > X, r = 1, 2, 3, • • • 
and that the integrals 

(420 / dV{x), (r = 1 ,2, • • • 2m), 

have finite values, Then^ for any 5 > 0 

(43) lim = 0. 

navOO 

This lemma, on which the main theorem of Part II is based, will be estab¬ 
lished if it is shown that the formula (36) holds true for functions ip satisfying 
the conditions (42). 

In the transition from the complete expression (10) for the expectation En 
to the as 3 nnptotic value (25) two essential steps were made. First, certain 
products of the form (11) have been omitted and, second, certain products 
of as defin.'d in (19) have boon arbitrarily added. This was allowed be¬ 
cause each of the products was seen to be smaller than 1 and their number was 
of the order 0(?i"‘“^). If a quantic in integral form ( 6 ) is considered which 
involves an infinite number of expressions like (10), a sharper estimate is 
necessary. 

It is easily seen that each integral (11') is a polynomial in including the 
product py}py 2 'Pys * ■ • and another factor which is certainly bounded whatever 
the pyK are. Thus, if the expectation of ti {2 • • * fcm is computed, each term of the 
form (11') consists of a finite factor and the product pyipyz • • • py, 2 m • In passing 
to the expectation of the quantic, the have to be replaced by dVy{x^) and 
each neglected term in (10) leads to an expression like 

(45) J*J ‘ ' J > ^2 > * * ’ > ^ 2 m) dVyy{X^ dVy^{x^ • • • dVyx{Xi^. 

According to the assumptions of Bs this integral has a finite value. The num¬ 
ber of neglected terms being of the order 0(71*"“^) the omission of these terms is 
justified. 

On the other hand, products of equal, except for the sign, products 
of py^pyx as long as I K and, except for a finite factor, products of as often 
as I = K. Again it is seen that the arbitrarily added terms sum up ho integrals 
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of the fonn (45). This shows that here too, if the conditions of Bs are fulfilled, 
the procedure leading to (25) may be applied. 

It follows that, under the conditions (42), if the integral (42') has a finite 
value, eq. (36) is correct and (43) is an immediate consequence of it. On the 
other hand, it is obvious that weaker conditions than those given in would 
suffice to establish (43). 


6. Theorem on products of n functions. The principal source of all explicit 
formulas on asymptotic distributions lies in certain properties of products of a 
great number of factors. Laplace devoted a part of his fundamental Treatise 
of Probability to these problems, but a complete outline of all results from a 
modem point of view is still lacking. In the third part of the present paper, a 
rather simple statement on this line will be used which may be formulated here as 
Lemma C. Let Fp(zi , 20 , • • * , 2 *), (*^ = L 2,3 , •• •)ybe a sequence of analytic 
functions of k complex variables and Gn the product F 1 F 2 * • • Fn . Suppose that 
at the point Zi = Z 2 = ••• Zk = 0 all F, have the value 1, vanishing first derivatives, 
and the second derivatives 



uniformly in each bounded region \z^ \ ^ Z in which the absolute values of the third 
derivatives of all F„ have an upper bound M, 

In fact, the Taylor development of F^ supplies under the conditions stated: 

(48) • • • , 2*) = I + i E + 0(Z^ 


and, therefore, 

(480 log F,(jsi , , • • • ,Zk) = hH A z, + O(z0. 

If here all are replaced by z,l\/n and the equations added for »/ = 1,2 , • • • ,n 
we obtain 


(49) 



Jl -- J- V 

Wn) “ 2n 


A[l^ZiZK + nO 



and this shows that the brackets on the left-hand side of (47) are 0(ZI'\/n ).— 
It is obvious that (47) would still hold if the condition concerning the third 
derivatives is replaced by a somewhat weaker one. 
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PART 11. DIFFERENTIABLE STATISTICAL FUNCTIONS 

1, Definitions. We consider a one-dimensional cumulative distribution func¬ 
tion V{x) as a point in the 7-space. If two points Vi{x) and V^ix) are given 
the functions 

(1) Vi(x) + t[V 2 {x) - 7i(x)], O^t^l 

represent the straight segment between Vi{x) and V 2 {x ). A subset of the 7-space 
that includes all segments determined by its elements is called a convex domain. 

Now, assume that a sequence of collectives with the distributions 7i(a;), 
72 (x), 73 ( 0 ;) , • • • be given. We shall consider functions f{V(jx)\ defined in a 
convex domain that includes particularly: (1) all average distributions Vn{x) 

(2) = - E V.(x) 

n v-i 

at least from a certain n on; (2) all repartitions Sn{x) that can occur, i.e. the 
repartitions of n quantities that belong to the label sets of the given collectives 
(e.g. positive x, etc.). If 7°(x) and V(x) are any two points of the domain, the 
quantity 

(3) F(0 = f{V\x) + t[V(x) - 7^(x)]), 

is a function of the real variable L It will be supposed to admit derivatives 
with respect to t up to the order r + 1. 

Following Volterra [9, 10] we define (in a slightly modified way) the derivative 
/' of a statistical function / in analogy to the set of partial derivatives of a func¬ 
tion of several variables. If V{x) would stand for a set of distinct variables 
7i, 72, 73 , • • • and 7 °(.t) for their initial values 7^,72,75, • • • one would 
have 


+ t{V{x) - F(x)]|,^ = (F, - Fj) 

dt V y 

where bj/b Vy is the partial derivative of/ with respect to Vy taken at the point 
7y = 7j . Thus we write 

(4) = / r[V\x), y]d{V - F")(y) 

and call /' which depends on Y^{x) and on a scalar variable y, but not on 7(x), 
the (first) derivative of/{7(a:)} at the point 7®(x). Only if a relation (4) is 
fulfilled for any two points of the convex domain, / is called a (one time) differen¬ 
tiable function. 

The derivative of a linear function 
A = J a(a;) dV{x), ^ 


( 6 ) 
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is simply the factor a(t/), P{y) ••• respectively, independent of the point at 
which the derivative is taken. If / is given as a function of A, J5, • • • one has 

(6) /'( V(x), 2/1 = ^ a(2/)i+ ^ + • • • • 

The derivative of the non-linear function 

(7) f=jj 4'(x, y) dV{x) dViy) 
is 

(8) r{y"(x), y\ = I [Hx, y) + ^{y, x)] d^x). 

Note that an additive constant in f (i.e. a quantity independent of y) has no 
significance since the integral of d(V — F®) vanishes. It follows from (6) 
that the first derivative of the mth order variance as defined in (2) of the Intro¬ 
duction, at the point V^{x) is 

(9) iy - oo)” -my fix- Oo)”-* dV^ix) 
where Oo is the mean value of V^{x), 

In the same way derivatives of higher order can be introduced. The second 
derivative of f{V{x )) is a function of F®(x), i.e. of the point at which the deriva¬ 
tive is taken, and of two scalar variables y, z which correspond to the two sub¬ 
scripts in the case of a function of distinct variables. The definition of 
f*{V{x)y yj z] is given in the equation 

+t{Vix) - F"(a;)])._o 

/"|F“(x), y, 2 ) d(y - V’>)iy) diV-V^iz). 

The second derivative of a linear function is zero. The function (7) has the 
second derivative y) + z) independently of The mth order 

variance gives, twice differentiated 

(11) -2mziy - + m(m - l)yz / (x - Oo)”^* dF“(x). 

The variables y and z in /" or in any additive term of /" may be interchanged 
and a term depending on one of them may be added or omitted. Thus, 
can always be written as a symmetric function of y, z without linear terms 
Accordingly, the second derivative of (7) is also 2^(y, z). 
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The derivative of rth order of / at the point V^{x) will be defined by the 
equation 


tj{V\x) + mx) - V\x)]]^ 

( 12 ) 

= //•••/••• ,yA d{V - d{V - y'’)(y,). 


Here, for given may be supposed to be a S 3 rmmetric function of the r 

variables 2/1 , 2/2 , * • • , 2/r • The rth derivative of the mth order variance is 


(13) 


(m — r + 1)! 


2/1^2 •••2/r 


X [(m - r + 1) I (x - 0 *)“-'’dy'Cx) - E 
In the case r = m the expression becomes independent of y®(a:), viz. 


(13') 


(-1)”*^! 2 / 12/2 • • • 2/m(l “ m) 


where terms depending on less than r of the variables 2/1 » 2/2 > * • • > 2/r have been 
omitted. 

If the definitions (4), (10), (12) are confronted one can see that/"{F, y, z\ 
is the first derivative of /' {F, 2 /1 etc. For proofs see [9] and [10]. 


2. Taylor development. The function F(t) defined in (3) admits the develop¬ 
ment 

(14) F(l) - F(0) = F'(0) + iF"(0) + ••• +J-,F<'>(0) + F''+"(,» 

where d is some quantity between zero and one. According to (3) the left-hand 
side equals the difference/! F {x) ) — /{ V^{x) }. The expressions F'(0), F"(0), • * •, 
are the derivatives as defined in eqs. (4), (10), (12). In the last term 
to the right, one has to introduce the distribution 

(15) . F(x) = ■F®(x) + «?[7(x) - y®(x)] 

and then to take the (r + l)8t derivative of / at the point 7'(®). 

For a given V^(x) each one of the terms on the right-hand side of (14) is a 
function of V(x). Except for the last one—^in which ^ depends in a certain way 
on F(x)—they are qmntics with respect to F(x) — F®(x), of the same kind as 
those considered in Part I. (There we had Sn instead of F and Fn instead 
of F^. 
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The rth term of (14) can be written as 

(16) // ••• >==r) d(V - V')(x^ ■■■ d(V - V^(Xr) 

where according to (12) 

(160 ^(xi ,%,•••, X,) = /''’{7®(x), xi, xa, • • • , x,}. 

To find the characteristic properties of Fr we compute its derivatives at a point 
Viix), To do this we must replace in (16) the V(x) by 

Viix) + t[V{x) - V^{x)] 

then differentiate the product 

(17) n d[{Vi - T^(xJ + t{V - 70 (x,)] 

with respect to t, and finally set ^ = 0. The derivative consists of r terms 
the first of which will be 

d(F - 70(x0nd(F, - F'’)(x.). 

ic-2 

Due to the fact that ^ may be supposed as a symmetric function, all r terms 
supply the same integral. Thus the derivative of Fr with respect to t at the point 
< = 0 can be written as 

Compaxing this with the formula (4) which defines the first derivative of a 
statistical function and writing y instead of x and F(x) instead of Fi(x), we find 

Fr{V{.x),y] = 

(18) 

(T^! // ■ ■ ■ / - ,xr)d{y - F*)(x0 • •. d(F - F“)(x0. 

This is the first derivative of Fr{y(a:)} at the point V{x). It vanishes at the 
point V{x) = V\x)» 

The integral in (18) has the same form as that in (14) except that its multi¬ 
plicity is (r — 1) rather than r. Thus it is immediately seen how the higher 
derivatives of Fr can be found. For the second derivative Fr{V(x)f y, z] 
we have simply to replace (r — 1)1 in (18) by (r — 2) I, then Xa by z and finally 
to omit in the product the differential d(y — ^(aja). This procedure can be 
continued up to the derivative of order (r — 1). The rth derivative, finally, 
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will be 

(19) 2/1, ^2, • • • , 2/r} = > 2/2 > • • • , 2/r) 

independent of V(x) and, according to (16'), equal to the rth derivative of 
f{V(x)} at the point V^{x), It is also seen that all integrals of the fonn(16) 
or (18) vanish if V{x) equals V^{x). The results can be summarized as follows: 
The sth term, (a = 1, 2 , • • • r), of the development (14) is a function of V{x) 
for which all derivatives at the point V^{x) except that of order a vanish while 
this one equals the ath derivative of the original function f{V(jx)} at V^(x), 
The complete analogy of (14) with the Taylor development of a function of 
distinct variables is thus evident. 

If we assume that f{V{x)} is a function whose first (r — 1) derivatives vanish 
at the point 7®(a;), eq. (14) takes the form 

y(x) - V\x) = i // ... jf'nv\x), 2 / 1 , y,, • • • , y,} 


■d(V- y®)(yi) ••• d(F- y*)(y,) 

( 20 ) 

+ - , 9^.1 

■d{V- y*)(yO F*)(yH-i). 


By applying to this formula the lemmas A and B of Part I, we shall arrive at 
the general theorem on asymptotic distributions that is the principal goal of 
this paper. 


3. General theorem. The main result to be derived in the general theory of 
asymptotic distributions is that the so-called normal distribution represents 
the first element in an infinite sequence which includes the asjrmptotic dis¬ 
tributions of all differentiable statistical functions, except certain irregular 
cases. The Gauss distribution covers in fact only those functions whose Taylor 
development starts with the first (linear) term, in particular the linear statistical 
functions themselves. If the first (r — 1) terms in the development vanish, 
the asymptotic distribution of type r becomes valid. 

Theorem I: Let Fi(x), 72(x), 78(x), • • • bean infinite sequence of distributions 
and f{V{x)\ a statistical function with derivatives up to order (r+1). Denote by 
Sn{x) the repartition of the n label values in the collective with the distribution element 
dVi{x)^ dV^ix) • • • dVr,{x) and by 7n{x) the arithmetical mean of 7i(x), 
Vi{x) , • • • , Vn{x), If for large n the first (r — 1) derivatives of /{7(x)} at the 
point Vn{x) vanish and the rth derivative equals ^n(2/i, 2 / 2 , * • • , 2/r), then the 
distribution of 

( 21 ) 


= n"V{Snix)) -nVnix)]] 
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is asymptotically equal to the distribution of the rth order quanlic 

( 22 ) If ' 

■d(S„ - ?n){xddiS„ - ?n)iXi) ••• d{Sn - Tn) (ov) 
under the following conditions: 

a) The distribution of (22) has a uniformly bounded derivative for all n; 

b) Within a convex domain in the V-space that includes all V^nix) from a certain 

n on, and all Srfx) that can occur, the (r + l)s< derivative of f{Vix)\ is smaller 
in absolute value than a prodiLct ^(yi)'^iy 2 ) • • * ^(2/r+i) whereby the 

integrals J [^(a;)]* dVy(x) for k = 1,2, • • • , 2(r + 1) have a finite upper 

bound /or = 1, 2, 3 , • • • . 

In order to prove this we introduce in eq. (20) Sn{x) for V{x) and Vnix) for 
y®(a;), and multiply both sides by Using the notations (21) and (2) and 
writing T„ for (*Sn — Vn ), the equation reads 

An- Bn 

(32) ^r/2 

(r + 1)! 

According to Lemma A the theorem will be verified if we can show that the 
expectation of the absolute value of the right-hand expression in (23) tends to 
zero. 

According to the Schwarz inequality one has, for any real C: 

(24) En{\C\} ^ 

For fixed values of F„ and Sn the integral on the right-hand side of (23) is a 
quantic of order (r -f 1) with the coefficients ^,+ 1 ( 2 / 1 , 2 / 2 , •••,2/hi)- The 
square of this integral is a quantic of order 2(r + 1) whose coefficients are a finite 
number (depending only on r) of terms each of which is a product of two ^r+i- 
values implying 2(r + 1) variables 2/1 > 2 / 2 , • * • , ^ 2 (r+i) - The absolute value of 
these coefficients is, therefore, according to the condition b) smaller than a 
finite factor times the product ^(yi) ^{yf) • • • 4'(2/2(r+i)) and thus fulfills the 
condition of lemma Bs. If the right-hand side of (23) is identified with C, the 
expectation of C^is, exceptfor a finite factor, the product of n'^ times the expectation 
of the above-mentioned quantic of order 2(r +1). It then follows from lemma 
Bj that the limit of En{C^\ is zero and from (24): 


limFnilCni} = limF7„{|An - Bn\} =0. 


This accomplishes the proof of Theorem I. 

If we apply here what was shown in Part I about the asymptotic distribution 
of a quantic, we can also state the following. 
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Theorem II; Under the conditions of Theorem /, the asymptotic distribvMon of a 
differentiable statistical function f{Snix)} is essentially determined by 
the average distribution fnlx ); 

(he first non-vanishing derivative of f{V(x)} at the point V'n{x)\ 
the average distribution excess 

Vnix, y) = ■pnW - - s yx^)v^{y), X ^y 

n v-i 

= Vn{y) - - i: V.{x)V,{y), x-^y. 

n 

By ^‘essentially determined*’ is meant determined except for an additional 
function whose moments of any order are zero. The statement then follows 
from Theorem I in connection with the fact that the asymptotic moments of 
quantics have been computed in Part I from the values of Unix, y). 

That functions with all moments vanishing exist has been known for a long 
time. A simple example given by Shohat and Tamarkin [6] is the following. 
Let /c be a positive constant smaller than i, and u = x\ k = tan kit. Then, 
the density (positive or negative) 

(26) <pix) = sin (ku) = Im 

fulfills the condition. In fact, the nth moment of (26) is the (vanishing) imagi¬ 
nary part of the integral 

(27) 1 f rf,, = (-^ 1!- ' (cos r (’^~). 

K Jo K \ K / 

Since <pix) takes negative values of the amount it can be superimposed to a 
given distribution density only in cases where the original density remains 
greater than some multiple of e~'* = exp i—x^). It can be shown that the moment 
problem is determinate (i.e. the distribution determined by the moments in a 
unique way) if the density vanishes at infinity at a sufficiently strong degree. 

From the standpoint of statistical theory two distributions with the same 
moments throughout may be considered as equivalent. This justifies the ter¬ 
minology used in Theorem II. On the other hand, Theorem I is independent of 
this restriction: The asymptotic distribution of the statistical function/{5n(a;)} 
is under the given conditions identical Avith that of the corresponding quantic 
of mth order. A detailed discussion of the case m = 2 will be given in Part III. 
Here follow some illustrations for the general case. 

4. Illustrations. The existence of asymptotic distributions of higher types 
can be exemplified in a comparatively simple way if we start from any known 
asymptotic distribution of a statistical function. 

Let us assume that g[Vix)\ is a function fulfilling the condition 

(28) ff{r,(a!)} = 0 


a) 

b) 

c) 


(25) 



330 


R. y. MISBS 


for all n, and that the asymptotic c.d.f. for ff{Sn(x)) is known. There will be 
some positive integer r such that 

(29) Prob [^{iSin(a;)} ^ 

K, for instance, gf is a linear statistical function r will be 1 and, under well- 
known conditions, ^>n(x) a normal (Gaussian) c.d.f. with finite variance depend¬ 
ing on n. 

Now, let / be an ordinary function of gr and thus another statistical function 
which may be denoted by /(V(x)}. According to the rules of differentiation 
we have 


(30) /'{V(x),u) 

dg 

and analogous relations can be derived for the derivatives of higher order. In 
particular, the following statement, valid in ordinary differential calculus, holds 
true: If P(a:)) has derivatives of every order and if the first s derivatives of / 
with respect to g vanish at some point g = g{Vi(x) } then also the a first deriva¬ 
tives of / with respect to V{x) will be zero at V{x) = Fi(a;). In this way we can 
devise statistical functions, with vanishing derivatives, for which the as 3 niiptotic 
distribution is known. 

For the sake of simplicity we may assume that (29) holds with r = 1 and 
that J{g) is a monotonic increasing function, given in the form 

(31) Jig) = g^{\ + a{g)\ 
with 8 a positive integer, and the inverse function 

(310 + m 

where fiij) goes to zero with / 0. Then, from (29): 

(32; Prob [/{^n(ic)} ^ ^niz') 

if z and z are connected by 

It follows that 

z' - ^ 0 


and if $»( 2 ') is supposed to be continuous, (32) becomes 
(33) Prob [/{;S,(a:) j ^ ~ #.(«*'*). 


This is a distribution of type a. 

Take as an example for g the arithmetical mean 


g\Sn{x)} = 


Xi -f- 0^2 + • • • + 


- 


(34) 


n 
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where Xi, , • • • , are the observed values and dn is the arithmetical mean 

of the mean values of Vp{x). Then, under certain restrictions for the 
there exists a boimded sequence l^n so that 

Prob[\/ri^ ^ 2 ] ^ £ e“*»*** du. 

Now if we choose 

f = 6(g -sin g) = »‘ ^1 “ ^ H-^ 

the asymptotic distribution of / will be given by 

Prob [n \/nf S z] du 

with the probability density 

-( 2 / 3 ) -»*.»/» 

3 Vtt * ® 

Similar examples can be drawn from the asymptotic distribution of nx if one 
asks for the distribution of appropriate functions of nx, etc. 

PART III. SECOND-TYPE ASYMPTOTIC DISTRIBUTION 

1. Statement of the problem. We now propose to study the asymptotic 
distribution of a quantic of second order as defined in eq. (6) of Part I. It 
has been shown in Part II that this covers the case of any statistical function 
of which the first but not the second derivative at the critical point vanishes. 

Independently of what was said before, the problem can be stated in the fol- 
lomng way. Given a function y) and a sequence of cumulative distribu¬ 
tion functions Fi(a;), ^ 2 ( 0 ;), Vz(x) • • •. Let Y«(a;) be the arithmetical mean of 
Vi{x)f V 2 {x) , • • • , Yn(a;) and Sn{x) the repartition of a sample zi, 22 , • • * , 
drawn from the collective with the distribution element dVi{z^ ^^ 2 ( 22 ) , • * * > 
dYn( 2 »), that is: nSn(x) is the number of those of the observed values 
Zi, Z 2 , •' • y Zn that are smaller than or equal to x. Then the quantity 

(1) f j f 2/) dTn(x) dTniy), where Tn(x) = Sn(x) - Vn(x) 

is determined by the observations zi, Z 2 , • • • t Zn . We ask for the distribution 
of / at large values of n. 

Without loss of generality, the function ip(x, y) can be supposed to be sym¬ 
metrical. If, in particular, y) = ^(a:)^(y), the quantity / becomes the 
square of 

1 4>{x) dTn(x) = ^ 


( 2 ) 
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and its as 5 mfiptotiG distribution can be computed in the manner shown in the last 
section of Part I. Another example would be 

ix ^ y) 
{x ^ y). 

In this case, integration by parts shows that 

(4) f{Sn(x)} = f dx 

where g' is the derivative of g. This is the statistical function that takes the 
place of in continuous problems. See Introduction eq. (3). 

Note that the ^'excess” 2\(x) vanishes at a; = dh « and that for sufficiently 
large x the increment dTn(x) equals —dV^nix), Thus, conditions for the exist¬ 
ence of the integrals in (1), (2), (4), etc. can be expressed in terms of the given 
functions ^(a;, y) and Vv{x), 

We shall first study the special case that implies so-called discontinuous chance 
variables. In our terminology it is the function \l/(x, y) that has to be specified. 
Let /i, / 2 , • • • , /ji be A; mutually exclusive one-dimensional intervals (or groups 
of intervals) and /*+! their complement. Assume that y) has a constant 
value when x falls in /* and y falls in J«, (t, k = 1,2, • • • , A; + 1). The increments 
of Sn{x), Vn{x), Tn{x) in the interval will be called p«, , f* respectively. 

Clearly, np« is the number of observed values falling in , np« is the expected 
number of such values, and n(p« — p«) = the excess of observed over expected 
numbers. Note that the given distributions y,.(a*) determine increments 
in the interval Z* and that 

(5) P«c = - (Pl* + P2« + • • • + Pnit)- 

^ % 

Since the sum of all J« must be zero we can replace {*+] b > 

(6) Ji+i = —~ {2 — —fib. 

Thus, the integral (1) can now be written as a sum of terms 

(7) /{&(*) 1 = zVui.i. 

like that introduced in the second eq. (8) of Part L 
Our next task will be to find the asymptotic distribution of (7) which depends 
on the matrix ( 4 , k = 1, 2, • • • , Aj), and on the succession of probability 

values Pf« I C*' = 1, 2, 3 , • • • ; k = 1, 2 , • • • A;). The matrix in k variables 

will be supposed to be symmetrical. 

2. Characteristic function. We define our chance variable as 

( 8 ) 


(3) 


y) = gKx) 
= g{y) 
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All summations, here and in what follows, are to be extended from 1 to k if 
not otherwise indicated. If Pn{x) is the c.d.f. of a:, that is 

(9) Prob|^/g®| = P„(a;) 
the characteristic function (c.f.) is defined by 

(10) Qniu) = = j e^“*rfP„(x). 

In order to compute Qn we assume that the quadratic form (8) is transformed, 
by a linear transformation, into a sum of squares. Using appropriate (in general 
complex) coefficients one can write 

(11) ^ = rj (vi + vl + ••• + vl)y = X • 

(The form is here supposed to be non-singular which, however, means no 

loss of generality). It will be seen later that explicit knowledge of the 
is not needed. 

Now, for any real or complex the identity holds: 

02) - vr. h"'*'' 

if we write v for V ui and replace in (12) successively y by vy/ n rji , v's/ n »? 2 , • * • 
we find 

(13) e^“' = (2 t)-‘'* jj J exp [-^E + vViiE^^.] dkdh ■■■dk 

where 

(14) ~ ~ (k = 1, 2, • • * , Aj). 

i 

Since the first exponential factor in the integrand is a constant with respect 
to the chance variable, the expected value of is given by 

(15) Q„(m) = = (2ir)-‘'“ II ■■■ I exp [-^E «=.] G„ dk dfe • • • dk 

with 

(16) Gn = E{ exp lv\/n 23 1. 

In order to find (j„ we consider the following n collectives Ci, C 2 , , Cn 

with discontinuous, {k + 1)-valued distributions: In C, the label values are 
, 22 , * • • , 2*, and Zk+i , with Zk+i = 0, their probabilities p»-i, , • * * , Pr.fc+i • 

The c.f. of this distribution at the point —ivl\/n is 
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If we multiply the n expressions (17) for v = 1, 2 , • • • n the product will be— 
according to well-known rules of probability calculus—^the c.f. for the distribu¬ 
tion of the sum of the n label components in the collective formed by combining 
Cl, € 2 ^ • • • , r„ . This sum is 

and therefore, 

(18) 

Multiplying both sides of this equation by 

and using the abbreviation 

(20) Zp = 

K 

we arrive at 

(21) Gn = E{ exp [vVn 13{.2.1) = E 1 F 2 ••• P» 
with 

A:+l , 

(22) 

Kwml 

This solves the problem: By inserting (21), (22) in (15) and carrying out the 
integration with respect to ^, • • • , 4 one has expressed Qn{u) in terms 
of the given and of the coefficients which link the z,, to the . This ex¬ 
pression for Qn{u) holds for all u. 

We have still to show that the integral (15) exists, at least for small | w | or 
I a 1, independently of the value of n. For this purpose we develop , as given 
in (22), in the neighborhood of t; = 0. At this point == 1 and the first deriva¬ 
tive vanishes by virtue of (20). We thus have 

2 A+1 

(23) = 1 + f Z Pr,(z. - z,y 

Zn K-i 

with 1 ^ 1. From the definition of 2 , in (14) it follows that the ratio | 2 . \/T 

with 

T* =» + ^2 + • • • + tl 

has an upper bound depending on the «« only. On the other hand, according 
to (20), z, is a weighted mean of the 2 , and, therefore, 1 2 , — 2 J will not surpass 
twice the maximum | 2 , |: 

(26) 


\z,-2p\ <aT 
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where a is a positive function of the coefficients a** which, in turn, are deter¬ 
mined by the . Introducing (25) in (23) we find 

I I < 1 + g 

2n 

and, finally, from (21): 

(26) iGnl < c'*'‘*“*^* = gl«la2r*^ 

Thus it is seen that for 

(27) I ^ ^ ^2 or 1 - 2a I w 1 ^ j/" X) 
the integral (15) admits the upper bound 

(28) 1 Qniu) I < (2^)-*'^ jj ■■■ f e dh,dl,, dk = n \ 

It also follows that the coiitrib\ition to Qn(u) from the region T > To tends to 
zero ith increasing To , uniformly with respect to n and with respect to u in 
the region \u \ < l/2a. 


3, Asymptotic value of Qn(u), If the quantity F„ introduced in (22) is con¬ 
sidered as a function of Zi/y/ n, Z2/^/ n, • • • , Zk/^/ n, we may write 

fc+i 

(29) F,(z,,zu-- - ,Zk) = EP»e”'*'"*'’. 


Here, Zp is defined by (20) and, on the right-hand side, Zk+i is zero. These func¬ 
tions Fp(zi , Z 2 , Zk) fox V = 1, 2, 3 , • • • have all the properties required 
in Lemma C of Part I: At the point zi = 2:2 = • • • = 2 * = 0 one has Fp = 1, 
the first derivatives are 

dFv ^ ^ 

_ = vvv, - vpp, 2 ^VyK=o 

dZi K«i 


and the second derivatives, ( 19 ^ k), 


(30) 


dJF, 

dz^ 

dW, 


dz, dz, 


? r fr+i -1 

"2 = jPi<i) V pvi P>'« J 

«*?..(-?J - |^p« - p« E Pf^J = 


= v'Pf.CI - P»f) 
= — »Vf.Pw 


The third derivatives are certainly bounded in any finite region of the 2-space, 
and this means also in any finite region of the <-space. 

The matrix of the second derivatives except for the factor is exactly that 
defined in eq. (19) of Part I: 

(31) ~ PwiPpK 
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and the arithmetical means of the derivatives from the matrix in eq. (23) of 
Part I: 


(310 


1 n 1 

= - Sp.'.S*.-23 Pi 


n 


n 


Applying TiCmma C we find 


(32) 


Gn =Gn 


^2 


Zk \ 
Wn) 


' exp 


[2 


This is valid in any finite /-region. Since it has been shown at the end of the 
foregoing section that, for small 1|, the outside contribution to the integral 
(16) converges uniformly (for all n) towards zero, we are allowed to introduce 
(32) in (15). Writing 


(33) 


23 = 23 7i« ^ , whereby 7 ., = 23 Pxm 

l.K l,K \,H 


equation (15) becomes 

(34) (?»(«) /exp[^-+ 


Now, it is well known that if nitK is any positive definite matrix with the de¬ 
terminant I I, then 


(35) (27r) jj • * * / ^xp 23 dti dk-' dtk = • 

This is likewise true if the matrix m**, which we also call My has the form M = 
Ml — XM 2 where Mi is positive definite, Mz arbitrary (complex) and [ X \ suffi¬ 
ciently small. Thus, the integration formula (35) applies to (34) and the result 
is reached, for small | w 1: 

(36) Qn{v) Q{u) = = I - X7t« I. 


If the a*, which transform the given quadric into a sum of squares are known, 

(36) with (33) supply the solution of our problem. 

The formula (36) is susceptible of several useful transformations. Let us 
write A for the matrix , A' for the transposed matrix, and 4^, P, F, I respec¬ 
tively for the matrices , P**, 7 j«, Then, obviously 

(37) ^ = A'A, r = ATA'y M ^ I - uiT. 


If we multiply M by A' to the left and by A to the right, we obtain 

(38) A'MA = A'lA - ui A'ATA'A = ^ 
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In this operation the determinant of M is multiplied by 1 Thus Z)(X) 

can be written as 

(39) D{\) = ^vith y;. = 

lr«| X,/i 

Here, the knowledge of the is no longer required. 

If the matrix (38) is multiplied twice by the inverse of we find — uxP 
and, therefore, 

(40) Z)(X) = X - XP.eI. 

As P is positive definite and real, it follows that all roots of D{\) — 
the **Eigenwerte** of T —are real numbers. Therefore, is a regular 

function along the real axis in the w-plane. Thus, (36) which was proved so 
far for small | u j only remains valid for all real values of u: The c.f. of the 
asymptotic distribution is represented by for all real w-values. 

Multiplying (38) only once by w^e obtain one of the two forms 

(41) I — ui or I — ui P^ 

which lead to 

(42) I^(X) = I \SiK I = I 5ti{ ““ XSki I , StK = • 

M 

Although this formula has been derived by means of it can be seen by con¬ 
tinuity considerations that it remains valid whatever the (symmetric) matrix 
\|/^g is. The formula makes it clear that the asymptotic distribution of the 
quadric 2^is completely determined by the ‘‘Eigenwerte^' of the matrix 
S = This bears out our second main theorem in Chapter II, as far as 

quartics of the form (8) are concerned. It will be seen in sec. 5 how (42) applies 
to the continuous case. 

We, finally, apply to (36) a transformation that is valid only if P has an inverse 
matrix P*. (As shown in Part I, sec. 4 this is not the case if the k intervals to 
which the subscripts 1,2, • • • , A; refer cover the whole range of the variables 
Xiy X 2 y • • • , Xn). Multiplying (41) by P* we find the matrix P* — ui^ and 
thus 

(43) D(X) = I P.. 1 X 1 Ptt - X^.. |. 

This is equivalent to 

(44) Q{u) = 1 P.. r'" j[ ■■■ / exp [- 

/ + dfid£2 * * • d^k . 

According to the definition of the characteristic function eq. (44) can be inter¬ 
preted as stating that 

(45) 


|P.* I* exp [“pP* £t£j 
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is the asymptotic probability density for the simultaneous occurrence of , 
f 2 , •**,?*. The expression (45) can be arrived at by applying the Central 
Limit Theorem to the case of k independent chance variables. Since, however, 
P* does not exist in general, eq. (44) would not be a suitable point of departure 
for developing the theory that concerns us here. 

4. Asymptotic value of Pn(jc), illustrations. The relationship between the 
c.f. and the c.d.f. of a distribution is well known and need not be disciissed here 
in detail. We shall use, in this section, two aspects of this relationship only. 
First, the continuity theorem, first proved by G. P61ya [5], stating that if the 
c.f. Qn{u) tend towards a limiting function Q(^^), the corresponding c.d.f. Pn{x) 
tend towards the P{x) that corresponds to Q{u). Second, the additivity, i.e. 
if Q{u) is of the form aQ'{u) + with a + /? = 1, then P(x) is 

olP'{x) + /3P"(a;) with the P'(ar), P"{x) corresponding to Q'{u) and Q"{xi) 
respectively. The following three groups of examples will illustrate the applica¬ 
tion of the foregoing results. 

a) Let us first consider a function of two excess values , {2 only 


(46) x = -/ == - (A{? + 1 {2 + Cf-i) 


where the matrix ^ is given by = A, = 4^21 = 4^22 = C. The product 

matrix P4^ is 


APii + BPi2 
AP21 + BP 22 


and the determinant of / — XP4^ 


BPii + CPl2 
BP21 “h CP22 


(48) D(X) = 1 - X[APn + 2BPu + CP 22 ] + AAC - B^)(PnP 22 - PJa). 

If Xi, X 2 are the two real roots of D(X) = 0, the asymptotic probability density 
of X will be 


(49) 


dP(x) 

dx 


du 




We are particularly interested in the case that P is “complete,” i.e. a matrix 
with all horizontal and vertical sums vanishing. Then Pn = Pi 2 = P 22 = , 

the last term in (48) cancels out and the only Eigenwert is Xi = 
1 /{A — 25 + C)pip 2 • Here, instead of (49) we have 

-Xix 


(60) 


dP{x) ^ 
dx 


= 1 f ^ ./Xi i_ 

r ^ X. 


m 

Xi 


This is, with respect to \/\x\ a Gauss distribution with the variance 
I A — 25 + C I PiP2/2* 
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If, in addition to the assumption that P is ^‘complete*’ (i.e. in the present case 
that 'p^i + Pk 2 = 1 for all v) the further assumption is made that the two inter¬ 
vals 1 1 and 1 2 cover the whole range of the original chance variables , 

Xit ••• , one would have also ti + {2 = 0 and from (46) 

X = I u - 2B + Oil. 

In this case, v^| a; | is a linear statistical function and the Central Limit Theorem 
leads to the same result as that expressed in (50). It is seen, however, from our 
derivation, that (50) holds under wider conditions: If p,i + p „2 = 1 for all v, 
there may exist another interval /* within the range of the chance variables 
Xi, X 2 i xz i • • • so that fi + {2 is not necessarily zero. 

The latter remark suggests the following general theorem: If / is a function 
of the k variables f 1 , { 2 , • • • , f* and g another such function but vanishing when 
f i + {2 + * * * + & = 0, then / and f + g have the same asymptotic distribution 
provided that for each v the sum Pt,i + + • • • + = 1. In the case of 

quadrics this result is equivalent to the following matrix theorem: If P, A 
are symmetric matrices, P with all horizontal and vertical sums equal to zero, 
^ arbitrary, and A of the form a,* = -f o« then the two products 

(51) and P(^ + A) 

have the same characteristic roots,—^This can be proved by the usual methods 
of matrix calculus. The matrix PA has all characteristic roots equal to zero.^ 
h) In the definition of Karl Pearson^s test function which is usually called 
X^, it is presumed that a sample is drawn from the combination of n equal dis¬ 
tributions. In this case all are equal and coincide with P which then can 
simply be written P: 

(52) Pt* = - p,p ,. 

The chance variable we now consider wdll be 

Thus = 5iic/p» and the elements of are 

(530 = Z = «« - Pi- 

A* 

The matrix I — has the elements 

5n(l — X) + XPi • 

If the fcth column is subtracted from any one of the others, only two terms re¬ 
main, one equal to 1 - X and one equal -(1 - X) in the last row. Thus, the 

•A proof of the matrix theorem has meanwhile been published by Alfred Brauer, Bull. 
Amer. Math. Soc., Vol. 53 (1947), pp. 605-607. 
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determinant D{\) includes (k — 1) times the factor (1 — X). On the other hand, 
D{\) is of degree {k — 1) and has the absolute term 1. Therefore 

(54) D{\) = (1 - \r\ 

This supplies the x^-distribution with (A: — 1) ‘'degrees of freedom 

_ ffP(x) 1 

Again, our result is slightly more general than that reached in the usual theory. 
It includes the case that in addition to the k intervals with the probabilities 
Pi )V 2 , • • * iPk (whose sum is 1) there are other intervals with probability zero. 
On the other hand, if to x^ a teim of the form n^(a, + is added, this 

would not change the asymptotic distribution. 

One may ask for other quadratic functions of , fo, * • * ,6 whose asymptotic 
distribution is given by (55). In particular, one might be interested in a generali¬ 
zation of x^ for the case of unequal original distrihuiions. The answer can easily 
be given by introducing the cofactors of order (A; — 1) and of order (A; — 2) of the 
determinant | Pik | . It was mentioned in sec. 4 of Part I that all cofactors of 
order (k — 1)—in the case of “complete’’ P —have the same value. It may be 
denoted by A. The cofactor corresponding to the lines t, k and the columns 
X, M will be denoted by TTuiX/* with 11 = 0 if t = /c or X = /x. Then, if I is any one 
of the integrers 1, 2, • • • , A; 

(56) ~ ^ II»i;kI 5 iy k ^ I 


is one possible solution. In fact, the product has in this case the elements 
(Pl^)*« = 6**, for ly K^l 


(57) 


= — 1, “ t = Z, K 4^ Z 
= 0, “ K = Z 


The determinant of / — XP^ is then seen to equal (1 — X)*“\ 

The solution (56), however, is unsymmetrical in the sense that it does not 
include any terms with . A completely symmetrical solution in which all 
f play the same role is given by 


(58) 


1 ^ 


According to (57) the matrix now consists of terms (fc — l)/fc in the prin¬ 
cipal diagonal and — l/A at all other places, that is 

1 

k' 


(580 
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In the same way as in the case of (53') it can be seen that the determinant of 
I — equals here (1 — X)*”\ The asymptotic distribution of with 

the coefficients (58) is, therefore, the 'distribution with {h — 1) degrees of freedom. 
If the formula (58) is applied to the case of equal the corresponding 
quadric becomes 




that is, + a term vanishing with + {2 + • • • + . One can easily modify 

(58) so that it leads to without any addition. 

c) A third group of examples where the asymptotic density is expressed by 
simple functions is that where D{\) is an exact square, that is, all characteristic 
roots (except the one that is zero) have even multiplicities. Let us assume k = 
2m + 1 and let Xi , X 2 , • • • , X;„ be m double roots. Then 


(59) 



with 


and therefore 


(60) ^ 

dx 

Assume, for instance, that all original distributions are uniform, that is 


p(>') 

tK 


= P« 



1 


and that the quadric/is given in the form (11) with the following : 

«•« = for I = 1 

= ’’ t > 1, /c = 1, 2, • • • , ft — 1 

( 61 ) ^ 

= - (i - l)Vkc^ ” t > 1, * = I 

= 0 , ” I > 1, K = t + 1, t + 2, • • •, fc. 


Then, the 7 ** as defined in (33) become 

7i« = c*i(ft — l)5i« 



for ft or K > 1 




= ic = 1 


and D{\) according to (36) takes the form 

k 

(63) D{\) = I - Ay.. I = n [1 - Ac.»(* - 1)J. 
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In other terms, for the quadric 

/ = /rciCfi + • • • + Jfc)* + A;c2({i — { 2 )^ + AjcsCJi + {2 — 2j3)^ + ... 

+ kcklii + {2 + • • * + {jfc-i — (A; — 

the characteristic X-valUes are 1/Ctt(t — 1). 

Now, to obtain the case of m double roots with A; = 2w + 1 we have simply 
to choose 


C 2 = SCi , 3C4 = 5C5,5C8 = 7C7 , • • •. 

The first term on the right-hand side can be entirely omitted in accordance to 
what was said in connection with (51). Besides, for the same reason, the ex¬ 
pression can be simplified in various ways by assuming fi + & +•••+{;& = 0. 
As a numerical example,’take A; = 5, C 2 = 3, Cs = 1, c = 5, cs = 3. Then 

/ = 20(5? + + 20 + 20 - {xfe - Hi - Ui + 10 SA) 

leads to the characteristic values X = 1/6 and 1/60 and the asymptotic density 
becomes 


dx 54 

In a similar way other groups of quadrics with asymptotic distributions of 
the type (60) can easily be constructed. One may, for instance, use eq. (41) 
and make vanish, in the matrix S = 7^, all elements on one side of the diagonal 
so that the roots are immediately known. 


5. Transition to the continuous case. In this concluding section, the transi¬ 
tion to the case of a quadric of the form ( 1 ) with continuous ^ (x, y) will be 
outlined. The formula best fit for this purpose is eq. (36). We therefore 
suppose the statistical function / given as 

(64) f = j j y) dTn{x) dTniy) with yp{x, y) - j «(»•, x)a{r, y) dr. 

In analogy to (33) we derive 

7 (®, y) == jj «)«(!/»<) dU„(s, t) 

(65) 

= J a(x, s)a(y, s) dV„(,s) — j J a(x, 8)a(y, t) dW„(8, t). 

Since dTT is symmetric, this function y(x, y) is symmetric with respect to x and 
y. If Z)(X) denotes the Fredholm determinant of the “kernel” 7 ( 3 ;, y), we con- 
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elude from (36) that the characteristic function of the as 3 rmptotic distribution 
of / will be given by 


( 66 ) 


Qniu) 


_1 _ 

D{ui) 


if certain convergence conditions are satisfied. 

In order to establish (66) the main point is to find a sequence of functions 
^i(^» y)y y)y * * * ^ach of thc type considered in the foregoing Sections and 
such that 1) the distribution of the quadric fk with the coefficients tends to¬ 
wards the distribution of / with increasing k and independently of n; and 2) that 
the determinants Dk corresponding to \l/k converge towards Z) as A; increases in¬ 
definitely. Using our Lemma A we can replace the first condition by asking 
that the expectation oi\f — fk\ should go to zero with k —> «independently of n. 

The following assumptions shall be made concerning / and the Vp(x): The 
function a(r, x) in (64) is continuous and bounded in every finite region; there 
exist two positive continuous functions a(r), ^{x) such that 


(67) 


I a(r, x) I ^ a{r)0{x) 


and that the integrals 


(68) f a\r) dr = M, j 0{x) dV,{x), j ^(x) dV,(x) 


exist, the latter two being bounded and converging uniformly with respect to 
V. We are going to devise a step function y) so that for the corresponding 
fk and any positive €i 

(69) £| 1/-M } ^ *1. 

Let N be an upper bound of the integrals 

(70) / 0{x) dV,(.x) ^N, j m d?„(x) g N- 
and € = €i/( 5 + 8 W). Choose a value L such that 

(71) f mdV,(x)^±, f 0\x)dVAx)g^ 

Z|x|>L M •'1»|>L M 

and, calling B the maximum of 0ix) in j a: | g L, another quantity R such that 



We subdivide, in the a:-j/-r-space, the domain 1 x | ^ L, ] y j g L, j r j g B in 
jfc* equal cells where k is determined by the condition that the absolute value 
of the variation of a(r, x)a{r, y) within each cell does not exceed €/4B. Outside 
this domain we set ^t(r, x) = 0 while inside the domain a*(r, x)a»i(r, y) shall 
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equal the value that a(r, a;)a(r, y) assumes in the center of the respective cell. 
Then y) will be defined by 

(73) i/k{x, y) = j ctkir, x)a*(r, y) dr. 

From the definition of k and from (67) and (72) it follows that 

I y) - Mx, y)l ^ I I air, x)a(r, y) - a*(r, a:)a*(r, y)\ dr 

JlrlS* 

+ f la(r, x)a(r,y)ldr 

•'lrl>K 

+ 0(x)0(y) [ a\r) dr ^ ^ + B* 


(74) 


■>‘_L 

2B* 


as long as I X I ^ L, I j/1 ^ L. If this square is called (L) and the comple¬ 
mentary region (L) we have 


(75) 


f - fk = f f U/(x, y) - ^kix, y)] dTnix) dTniy) 


+ ff Mx, y) dTnix) dTniy) 
J J(I) 


and since the integral of | dTnix) dTniy) I Is not larger than 4, while, according 
to (64) and (67) 

(76) 1 4/ix, y) I ^ 0ix)fiiy) j a\r) dr = M0ix)fiiy) 
we conclude from (74) and (75) 

(77) \f-fk \ ^4,+M [ [ m^iy) I dTnix) dTniy) \. 

This gives 

(78) E{\f-fk\] ^4^ + M [ [ 0ix)0iy)E{\dTnix) dTniy) 1). 

J J(i) 

Now, from | dT* | = | dSn — dV„ j ^ dr„ -|- 2dr» and from the formulas 
derived in Part II, 

E{ dTnix) } = 0, E{ dTnix) dTniy )} = - dunix, y) 

n 

it follows 


1 ,r-r 


E\ 1 dTnix) dTniy) 1 } ^ ; dCnix, y)4r4 dVnix) dVniy) 


(79) 
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with 

(79') dUn(x, y) = y) dVn{x) — dWn{x, y) ^ 6(a;, y) dV'ni.x), 

If this is introduced in (78) and (71) taken into account, we find 

E\\f-f,\\^ 4 t + M-l /3’(z) d?M 

tl J|x| > L 

(80) +4M [f fiixMy) d?.{x) dVjy) 

J J(I) 

^ 4« + 1 « + 4 X 2iV* g (5 + 8N)t = «i 
n 

as required in (69). 

On the other hand, it can be seen that the kernel 7(x, y) as defined in (65) 
is the limit of the sequence 7*(a;, y) 

yk(x, y)= ak(x, s)ak(y, t) dUn{s, t) for x, y in (i?) 

(81) ^ 

= 0 for X, y in (JR) 

where (R) means the region \ x\ ^ Ri\y\^ R and {R) the complementary 
region. In fact, from the definition of k and eqs. (67) and (71) one has for x, y 
in (R): 

1 yi.x, y) - 7*(*, ^ I dUnis, t) I 

+ f f 1 oi(x, s)a(y, t) dVnis, t) \ 

J Ja) 

(82) S ^ + a(x)a{y) T f ^{s) dVnis) 

+ -Eff dVM dVM 

71 vibI j •'(I) 

g ^ + a(x)a(2/) ^ (1 + m. 

Since o(z) is bounded, the right-hand side goes to zero with t. Finally, for 
X, y in (fi) we have 

1 y{x, y) — yk{x, y) | ^ // I a(x, s)a(2/, t) dUniSy t) 1 
g a(.x)a(y) [ j ^(s) dr„(s) 

+ -12 ff dV,(8) dy.w.l 

n Vii J J J 


(83) 
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Here, the two terms in the brackets are bounded, but a{x)a{y) goes to zero as R 
increases. The conclusion is that 'ikix, y) tends uniformly towards 7 ( 0 ;, y) 
with A; —> 00. 

Thus, eq. ( 66 ) is established provided that the function y{x, y) defined in (65) 
has a Fredholm determinant D{\) that is the li mit of th e corresponding alge¬ 
braic determinants and provided that the c.f. y/l/D{ui) leads to a c.d.f. with 
bounded derivative. 

As an example let us consider the case 

a(r, x) ^ y/ gr'(r) for r ^ a; 

(84) 

= 0 r <x. 


This function is not continuous as it was assumed in establishing ( 66 ). How¬ 
ever, the existence of a single discontinuity line, x = r, does not invalidate the 
argument. We assume g'(r) = 0 and equal to dg/dr. Then, in the case of 

(84) : 

(85) ^ ~ for X g 2 / 

= - g{x) “ X ^ y. 

Since, however, adding to ^ a function of x or of y alone does not change the 
value of /, we can also use 


(85') 


'I'fx, y) = g{,x) ioTx g.y 

= giy) “ x'^y. 


The statistical function / that corresponds to (84) can be computed either from 

(85) or (85')—or directly from (84) if we use the formula that follows from (64) 

( 86 ) / = /[/ Qf(r, x) dr„(x) J dr. 

The integral in the brackets is, in our case, seen to equal y/g'{r) Tn(r), thus 


( 86 ') 


/ = / g'(.r)[Sn{r) - r„(r)]“dr. 


This is exactly the test function w mentioned in the Introduction, eq. (3). 

To find the distribution of / we have to compute y{x, y). Its definition (65) 
can be written in the form 


(87) 7 (x, y) = ^ otix, s)a{y, «) dV,( 8 ) — j a(x, s) dV,U) ja(y, s) dy.(«) J. 
This supplies in the case of (84) 


( 88 ) 


y(x, y) = •\/^ix)g^{y)[y»(,x) — 7n(a:)F„(2/)] for x ^ y 

“ VjW^[r»(y) - r.(x)7„(y)] “ X ^ y. 
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Here, the second term in the brackets is the arithmetical mean of the products 

VXx)V.{y). 

If the distributions Vp{x) are all equal (independent of v) we have simply to 
write Vix) instead of Vnix) and V{x)V{y) instead of 7n(x)Fn(2/). If, in addi¬ 
tion, the distribution in the original collectives are uniform in the basic interval 
0 to 1 , one has 

(89) y) = V g'{x)g\y) a; (1 - y) for 0 g a; ^ 2 / ^ 1 

= \/g\x)g\y) y{l - x) '' 0 ^ y ^ x ^ I, 

This is the case dealt with in Smimoff^s papers [ 7 , 8 ]. If, finally, g\x) is sup¬ 
posed to be equal to 1 in the interval 0 , 1 , we arrive at a kernel y{Xf y) whose 
Fredholm determinant is well known: 


(90) ’■fc»)-»('-») '» o(x).!5^\ 

= y(l — x) “ x'^y. VX 

This supplies immediately the c.f. and (in form of a definite integral) the c.d.f. 
of the asymptotic distribution of t/ for g' = 1, 

The same result can be reached without the use of a(r, x) if we apply one of 
the transformations discussed in the foregoing Section. Take, for instance, 
instead of 7 (a:, y) the unsymmetric kernel <r(a:, y) corresponding to the matrix 
S = defined in (41). If all original distributions are equal, the element of 
S can be written as 

(91) Su = X = vMik — 

M M 

Calling v{x) the density dV{x)/dx in the continuous case, the corresponding 
kernel becomes 

(92) <r(x, y) = v{x) j^^(x, v) - j ^(s, j- 

With the ^-values from (85'), g' = 1, v = 1, this gives 


(92') 


ff(x, y) = X - 


+ |for 


L 

2 


X ^ y 
x^y. 


It can eaaly be seen that the “Eigenfunctions” of this <r{x, y) are sin(-\/Xm x) 
with = mV, and, therefore, the Fredholm determinant is that indicated in 
(90). 

It might be added that the expectation and the asymptotic variance of w 
can be computed, independently of the distribution, from the formulas de¬ 
veloped in Part I. The results are 


nEW\ = / ff'(a:)F„(x)[l - r.(x)l dx 


(93) 
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and, in the case of all F»(x) equal 

(94) nVar(«*l jf g'(x)g’(y)V\x)[l - V(y)]*dxdy. 

x^V 

These formulas have already been given in [4]. 

Another, more general, remark is this. If all V,{x) are equal, one can reduce 
the problem, by a transformation of the original chance variable x into x' = 
V{x), to the case of a uniform distribution over the interval 0 to 1. If the Vy{x) 
are not equal, it might still be possible to find a transformation x' = x'(x) such 
that all original distributions extend over a finite region on the a;'-axis only. 
In this case the restrictions concerning the behavior of the distributions at 
infinity drop out. 
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APPROXIMATE SOLUTIONS FOR MEANS AND VARIANCES IN A 
CERTAIN CLASS OF BOX PROBLEMS 

By Philip J. McCarthy 
Social Science Research Council 

1. Summaiy. Consider n boxes, each box having an associated probability, 
Vi t (S Vi = l)> associated integer, ki . If balls are thrown one by one 

i 

into these boxes, the probability being that any one ball falls into the ith box, 
then the number of balls which must be thrown in order to obtain, for the first 
time, at least ki^ balls in the t'lth box, at least fc*, balls in the i^th. box, • • • , and at 
least ki^ balls in the t,th box, is a random variable, A^,[A;i(pi), feCpa), • • • , kniVn)]- 
Here , ^’ 2 , * * * , u represent the numbers of that set of s boxes, (1 < s < n), 
which first satisfies the stated condition. 

The distribution of A^,[A;i(pi), fe(p 2 ), • • * , knivn)] can be written down for any 
set of values assigned to n, s, the p/s and the ki*8. However, for n greater than 
2 the distribution assumes such an extremely complicated multinomial form 
that except for certain special cases even the mean of the distribution cannot 
be numerically evaluated without a prohibitive amount of labor. 

This paper presents the exact moments of iVifA^iCpi), fe(p 2 )] and N^lkiipi), fe(p 2 )] 
in forms that readily lend themselves to computation and .shows how these 
moments can be used to obtain approximate values for the mean and variance 
for certain situations where n is greater than two. These approximation formu¬ 
lae are given for 

1. The mean and variance, for any n and any set of kiS and p/s when 8=1 
or n. 

The mean, for any n and 2 < s < n — 1, when p< = 1/n, ki = k, 
(i = 1,2, • • • , n). 

Some indications are given concerning the error of the approximations, and the 
circumstances which lead to a minimum (and maximum) error. Curves have 
been prepared to show the mean for the two box case, the primary function 
of these curves being to assist in the application of the approximation formulae. 
Some problems where the results of this paper might be applicable are suggested 
in the Introduction. 

2. Introduction. A box problem is defined when one is given a fixed number 
of boxes, a collection of balls (either finite or infinite), a set of rules governing 
the throwing of the balls into the boxes and a statement of the conditions which 
will bring the throwing to an end. The terminating conditions usually state 
either that a fixed number of balls will be thrown or that balls will be thrown 
until a particular distribution of balls in the boxes has been obtained. In the 
first of these, interest is centered on the possible distributions which can be ob- 
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tained, while in the latter the number of balls necessary to obtain a specified 
distribution is of primary interest. 

This paper will be concerned with certain problems falling in the latter cate¬ 
gory. In the simplest case one is given two boxes with associated probabilities 
Pi and p 2 and associated integers h\ and fe . Balls are thrown one by one into 
the two boxes, the probability being pi that any one ball goes in the first box and 
P 2 that it goes in the second box. This process is stopped when either hi balls 
fall in box 1 or balls in box 2, whichever occurs first. One is interested in the 
distribution of the number of balls necessary to terminate the throwing. This 
problem was stated in essentially this form by Laplace [4], but he contented 
himself with merely writing down the probability generating function. 

Here the special case of two boxes will be treated in detail and the results 
will then be generalized to the n-box case. In all of these instances it is pos¬ 
sible to write down exact expressions for the mean and variance of the number of 
balls required to achieve the stated distribution. However, in almost every 
case the resulting expressions are too complicated to be of any use when a numer¬ 
ical answer is desired. The principal portion of this paper will be devoted to 
obtaining approximate formulae from which numerical answers can be obtained 
for these problems. Some evaluation of the degree of approximation will be 
given in section 5, while curves to facilitate the computation will be given in 
section 6. 

The statement of these problems in terms of boxes and balls may lead one to 
the belief that th6y have no other interpretation. Actually this is not the case, 
and a few illustrations of this point will now be given. For example, consider 
the curtailed single sampling plan used in acceptance sampling. A buyer re¬ 
ceives a lot of articles. This lot will contain a certain proportion of defective 
items. The buyer wishes to determine on the basis of sampling whether to 
accept or reject the lot. His knowledge of his own situation will allow him to 
specify the largest proportion of defectives which he is ordinarily willing to 
accept and the risk he is willing to take of accepting a lot with a proportion de¬ 
fective larger than this critical proportion. On the basis of these two values it 
is possible to set up a sampling plan in which the buyer will take a sample of size 
n out of the lot, inspect it, and reject it if there are ki or more defectives in the 
sample. Of course once he has obtained defectives there is no need to inspect 
the remainder of the sample. The lot will then be automatically rejected. 
Similarly, once he has obtained n—ki non-defectives, he can accept the lot with¬ 
out inspecting the remainder of the items. The average number of items which 
he must inspect in order to reach a decision is given by the solution to the two 
box problem stated above. Box 1 will receive the defective items, the asso¬ 
ciated integer being ki and the associated probability being pi , the true propor¬ 
tion of defectives in the lot. Box 2 will receive the non-defective items, the 
associated integer being n—ki and the associated probability being p 2 , the true 
proportion of non-defectives in the lot, 

Laplace [4] considered problems of this type as applied to games of chance. 
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Thus suppose there are two players A and B who participate in successive trials 
of a given event, the probability being pi that A wins on any one trial and p 2 
that B wins. Then one can associate the integer ki with A and with B by 
saying that A wins the match if he wins ki trials before B wins fe trials and con¬ 
versely. The analysis is exactly the same as for the two box problem. It is 
apparent that this same situation can be extended to any number of players. 

Another possible interpretation is as a particular kind of random walk prob¬ 
lem. Let a particle start at the origin of a system of rectangular coordinates 
and suffer successive positive unit displacements, the probability being pi that 
it moves one unit in the a:-direction and p 2 that it moves one unit in the y- 
direction. Furthermore assume that it is absorbed if it ever reaches the line 
a; = A;i or the line y = . Then the analysis of the above two box problem 

gives the mean number of displacements before it is absorbed. In the same 
manner, such a random walk problem can be stated for n dimensions. For n 
equal to three, there will be three planes and the particle will be absorbed when 
it reaches any one of the three. 

Certain problems in public opinion polling may fit into this category of box 
problems, particularly if the above problem is rephrased so that one requires 
the mean number of trials to obtain at least ki balls in the first box and at least 
fe balls in the second box, for the first time. For example, suppose that one 
desires to sample from a population composed of two types of individuals, 
A and B. Let the population proportions of A and B be known and be de¬ 
noted by Pi and p 2 . Then if one wishes to obtain at least ki individuals of type 
A and at least fe individuals of type B, the average number of persons who must 
be chosen in order to fulfill this condition is given by the analysis of the cor¬ 
responding box problem. This is rather artificial when there are only two cate¬ 
gories and Pi + p 2 = 1. However, these restrictions will be removed in the 
course of the paper, and the problem will be considered for any number of types 
of individuals. 

As a final example, consider one of the many bombing problems which arose 
during the course of war research. Suppose that a factory which is to be de¬ 
molished has n vital units, the destruction of any one of which will destroy the 
usefulness of the factory. Let the probability be pi of hitting the first unit with 
a single bomb, p 2 the probability of hitting the second with a single bomb, etc., 
and assume that ki bomb hits will finish off the first unit, fe, the second, etc. 
Then the mean number of bombs required will be given by the analysis for the 
corresponding box problem. 

Corresponding interpretations are possible for the other problems which are 
to be considered in this paper. Some of these will be indicated as the analysis 
proceeds and it is to be hoped that others will occur to the reader. 

As previously noted, this paper will be concerned with the distribution of balls 
necessary to terminate the throwing, assuming the p's are known. Another 
possible interpretation is to assume the p's unknown and to estimate them with 
the results of the ball throwing. Certain aspects of this problem for two boxes 
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have been considered by J. B. S. Haldane [3] and Girshick, Mosteller and Savage 

[ 2 ]. 

3. Solution for the two box case. 

3.1. DistribiUion and moments of the number of trials necessary to obtain either 
ki halls in the first box or fe halls in the second box. This problem may be stated 
as follows: Suppose one is given two boxes with associated probabilities pi and 
Pi , and associated integers k\ and . For the present it will be assumed that 
Pi + ?>2 = 1, although tliis restriction will be removed later. Now let balls be 
thrown one by one into these two boxes, the probability being pi that a particular 
ball will fall in the first box and pi that it will fall in the second box. This 
process is stopped on the first ball which leaves either ki balls in the first box or 
ki balls in the second box. The number of balls, Xy which is required to accom¬ 
plish this is a random variable and we desire the moments of x. The probability 
that ki balls are obtained in the first box on the a:th throw, ki<x<ki + ki-- 1, 
before ki balls are obtained in the second box, is immediately seen to be 

Similar reasoning gives the probability that fa balls are obtained in the second 
box for the first time on the xih throw, h < x < ki + ki — 1, as 

(3.2) 

From (3.1) and (3.2), the hih moment of x, E{x^), is 

(3.3) L _ J) v\' pr “ + i: / (^1 _ }) pr“ p^. 

However, it is inconvenient to consider (3.3) directly. A much simpler pro¬ 
cedure is to determine the increasing factorial moments of x and then transform 
these into the ordinary moments. Thus the hih increasing factorial moment of 
Xy Fh,ilki{pi)y ki(pi)]y Is dofinod as E[x{x + 1) ••• {x + h — 1)]. Then Fh,i[ ] 
is equal to 


1 1 —fc* 

^IPl P2*. 

(3.4) can be transformed by means of the relationship 

22 + A pj s- (1 Ii^p^{k + 1, a + 1), 

y-o \ J / 


{x + h- 1) 
{x - 1)1 






»-*2 ““ 


l/x- 
\k2 - 



(3.5) 
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where /,(p, g) is the Incomplete Beta-Function as tabulated by Karl Pearson 
[6], and the result is obtained that 


(3.6) 


F r/ (n ^ 1 (n + 1) • • • (frl + — 1) 

rhAkiipO, 12{P2)] = - Ipiiki + h, h) 

k2(k2+l)---(k2 + h^l) 

1“ Ipt{k2 + a, fci)« 

P2 


The ordinary hih moment of x may be written in terms of Fi,i[ ], F 2 ,i[ ],•••, 
Fh,i[ ] as 

(3.7) E{x'‘) = S Fu [ ] ^\-1)*+‘, 

where A*0^ represents a difference of zero. Tabular values of A’OV^*! are given 
by Fisher and Yates [1]. 

In particular, the mean and variance of Xy which will receive the special desig¬ 
nations Ei[ki{pi)y k 2 (p 2 )] and <A[kiipi)y fe(p 2 )] respectively, are 


(3.8) ^ Ip,(Id + 1, fe) + - Ip,(h + hki) 

Pi Pi 

and 


(3.9) ^2 


Eilklipi) y k2{p2)] {Fi[A;i(pi), A;2(P2)11* • 


In the event the p^s arc equal and sum to one, Ei[kiipi)y ^(^ 2 )] will be abbreviated 
to Ei[ki y fe], and finally, if both the p’s and fc’s are equal, it will be written as 
Ei[k\ In this two box situation, the only other possibility is F 2 [A;i(pi), fe(p 2 )], 
which will denote the expected number of balls required to obtain at least ki 
in the first box and at least in the second box, for the first time. This problem 
will be considered in section 3.2. 

In order to facilitate the computation of mean values, both for the two box 
problem itself and for its application to problems involving a larger number of 
boxes, (3.8) has been graphed for various values of Aji , fe , Pi and p 2 . A dis¬ 
cussion of this procedure and the results obtained will be found in section 6. 

There is one further result which will later prove useful. Consider the situa¬ 
tion when there is only one box with pi and A^i, pi < 1. This is the same as 
having two boxes where the ^*2 corresponding to the second box is infinite. In 
other words, one can terminate the throwing of balls only because of what hap¬ 
pens to the first box, never because of anything that happens to the second box. 
In this case one obtains 

Ei[ki(pO, “ (ps)] = = 1^. 


(3.10) 



364 


PHIUF J. HcCARTBT 


Similariy, 

(3ai) « (pi)] = . 


3.2. Distribution and moments of the number of throws necessary to obtain at 
least ki balls in the first box and at least h balls in the second box. This problem 
may be stated as follows: Suppose there are two boxes with associated probabil¬ 
ities Pi and p 2 , and associated integers ki and • As in 3.1, pi + P 2 = !• 
Let balls be thrown into the boxes one by one, the probability being pi that a 
particular ball will fall in the first box and p 2 that it will fall in the second box. 
This process is stopped on the first ball which leaves at least ki in the first box 
and exactly fe in the second or at least fe in the second and exactly ki in the first. 
Again x is the number of balls recjuired to accomplish this. As explained in 
3.1, the mean value in this case will be written as E 2 [fci(pi), k 2 {p 2 )]- The analysis 
follows through as in 3.1 and the mean number of trials is equal to 


(3.12) 




(rO 






Pi Pi ■ 


Making use of (3.5), this can be written as 


(3.13) ^ [1 - + 1, W1 + - [1 - + 1, fci)]. 

Pi P2 

Referring to (3.8) it is evident that 

(3.14) Ei[ki(pi)j k2(p2)] + E2[ki(p^, k2{p2)\ = — + 

Pi P2 


The ^th increasing factorial moment in this problem, denoted by Fkjki(pi)f 
k 2 {p 2 )], is 




(3.15) 


Pi 


, ^2(A‘2 + 1) - » (^*2 + — 1) r T h 

+ h [1 ~ Ip2\k2 + hy All)]. 

P2 


Comparison of (3.15) with (3.6) gives the relationship 


^A,l[ J + Fk,2[ ] 

(3.16) 


Ari(Ari + 1) • • • {ki + A — 1) 

Pi 

. Ar2(A*2 + 1) • • • (Ara + — 1) 

"T 


The ordinary moments of x can be computed from (3.15) by the use of (3.7). 
That is, formula (3.7) holds in this case if F*,i[ ] is replaced by P’a, 2 [ ]. 
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It can be easily shown by the use of the recursion relationship for the Incom¬ 
plete Beta-Function, 

q) = - 1, g) H- (1 - x)h{pj g - 1), 

that Fa,i[ ] and FhA ] satisfy the partial difference equation 

Fh,i[HPl)yh{p2)] = hFH-i,i[kliPl)ykt{p2)] 

(3.17) + PiFhMi - l)(pi), Upz)] 

+ P2Fh,i[kl{pi), (fe — 1)(P2)], 

where i = 1 or 2. This equation can be used as an alternative way of obtaining 
many results, examples of which are (3.10) and (3.11). Certain of these appli¬ 
cations have been discussed by McCarthy [5]. 

4. Solution for the n box case. 

4.1. Preliminary discussion. The problems of this section, although direct 
generalizations of the two box cases, can perhaps be most easily stated and 
illustrated as applied to the behavior of a random particle. Suppose that 
we have a random particle which starts at the origin of n-dimensional rectangular 
coordinates and moves in unit steps along the positive coordinate axes. At 
any given point the probability will be taken as p* that it moves in the a;,-direc- 

n 

tion, ^ px is assumed to be one unless otherwise specified. Now consider the 

n hyperplanes, Xi = ki , and assume that the particle will be absorbed if it passes 
througli a specified number^ say s, of these hyperplanes. Notice that we are 
interested o^y in the number of planes which it passes through, and not in the 
particular ones. For each s, (s = 1, 2, • • • , n), the number of moves which the 
particle makes before it is absorbed is a random variable, and in this section we 
will be concerned with the distribution of this random variable. The cor¬ 
responding interpretations for boxes and balls is immediately obvious. 

These problems are seen to be generalizations of the t^vo box cases considered 
in section 3. Although it is always relatively easy to write down formal ex¬ 
pressions for the quantities to be considered, the step from two boxes to three 
or more boxes produces expressions which are extremely difficult, or even im¬ 
possible, to evaluate. In this section we shall develop approximate solutions 
which make use only of simple computations based on the solution for the two 
box case. 

As an introduction to the contents of this section, we shall discuss briefly a 
box problem which is a special case of the general problem. Assume that there 
are n boxes with a probability of 1/n that any one ball will be thrown into a 
particular one of the n boxes. Then one can ask for the mean and variance of 
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the number of trials required to obtain s occupied boxes (i.e, ki ^ ^ 

kn = 1). Making use of (3.10) and (3.11), we obtain 

Em = 1 

- >+4'“ G)] -'+^ 

EJ,+£..[, „(?)] 


(4.1) 


= 1 + 


n , n 
n — 1 n — 2 


^^.[11 = 1 + 


n — 1 


+ 


n — 5 + 1 


= nE— 

i-o n — t 


and 


Am = 0 


+ 


in - ly 


aiin = 0 + 


n 


(n - 1)* 


+ a! 


H^)> 


00 1 - 
W. 


(4.2) 


= 0 + 


+ 


2n 


(n-D* ' (n-2)* 


= 0 + 


n 


in - D* 


+ 


2n 


in - 2)> 

+ ••• + 


is — l)n 
(n - 8 + 1)* - " in - t)* • 




The solution for this problem for a = n is given in Uspensky [9], but a straight¬ 
forward solution requires a great deal of formal manipulation. The step-by- 
step procedure used here is somewhat indicative of the methods to be used in the 
succeeding portions of this paper. 

4.2. Mean and variance of the number of trials required to obtain either ki halU 
in the first box, or k^ in the second, •••, or kn-i in the (n — l)sf, the probability 
associated with the nth box being non-zero. The mean number of trials in this 
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particular problem is represented by • • • , « (p,)]. The 

formal expression for this quantity is 

^ ._ 0 1)1 jg^ 

, ^ (.ki- l)l(j - kd\^* 


where the third sum is taken over all values of the r’s such that 




+ + Ti+i + • • • + Tn = y 


Ti < ki y • • • j ri_i < ki^i , r,+i < fc»+i, • • • , ri,«i < fc»-i • 

This expression can be reduced by one dimension by the application of some of 
the results for two boxes. Consider for the moment only those balls going into 
the first (w — 1) boxes. Then the number of balls (conditional) which is neces¬ 
sary to obtain either ki in the first box, or fe in the second, • • • , or kn-i in the 
(n — l)st box is a random variable X which takes on values 

fci, fci + 1, • • • yki + k 2 + • • • + — (n — 2) 

with corresponding probabilities t,- , where with no loss of generality it is as¬ 
sumed that ki<k 2 < • • • < Ajn-i. tj is given by a sum of (n — 1) multinomial 

expressions, the probability associated with the ith box now being p*- ^ , 

which will be designated by p'i . 

Under these circumstances it is apparent that 

(4.4) Eilkiip^, , fc„-i(p,-i), « (p„)] = 23 v,Ei[xj(pi H-+ p,_,), 00 (p„)J. 

; 

However, (3.10) can be applied to each term in (4.4), leading to 

(p. + PS + ••• +P»-i) ? • 

Now from the definition of t, and Xj we have 

Eilkl(pi)y ••• ykn-l(pn-l), « (Pn)] 

= / , , \ Ei[ki(jf)i)y kf(pi)y ••• ,fei.i(p».i]). 

-r P 2 + • * * T- Pn-i; 

Similarly, the application of (3.11) gives the result that 

allkiipi), ••• , A;n-i(p»i-i), (Pn)j 

“ (Pi + Pf+^-" +P..-I)’ 
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These results are of immediate importance for two reasons: 

L They indicate that by combining boxes and introducing a new random 
variable, certain problems can be simplified. This statement will be expanded 
and the principle applied repeatedly in the later portions of this paper. 

2. With respect to the section on two boxes, they mean that the restriction 
Pi + P 2 = 1 is not necessary for the solution of the problems. One can always 
assume that psC = 1 ~ Pi ~ P 2 ) refers to a box which receives balls but which 
otherwise has no effect on the outcome of an experiment. In this paper it has 
been convenient to refer to such a box as having an infinite capacity. 

4.3. The mean value and variance of the number of trials required in a two box 
problem when one or both of the constants h and fe are replaced by random variables. 
The discussion in 4.2 has indicated that the idea of associating a random variable 
with a box instead of a single integer may sometimes lead to simplification. 
Here this procedure will be treated in more detail. Consider Ei[ki{pi)y fe(p 2 ^] 
and assume that ki is replaced by a random variable X which can take on values 
Xi,X 2 i • • • ,Xt with corresponding probabilities tti , • • • , 7r»-, • • • , 7r« . Under 
these circumstances Ei[ ] itself becomes the random variable j&i[X(pi), fe(p 2 )]i 
taking on values Ei[xi{p^^ ^ 2 (^ 2 )], {i = 1, 2, • • • ,0, with corresponding prob¬ 
abilities TTi . The mean value of this new random variable can be formally 
written down as 

(4.8) Emx(^d, fe(P2)]) = i: nEAxi{p,)M(:Pi)l 

This expression can always be calculated from the probabilities tt,* and (3.8) 
or from the curves given in section 6. However, in the applications which will 
arise later in this paper, this computation would be very time consuming. In¬ 
stead, an approximation to (4.8) will now be derived which will prove to yield 
very good results, and which can be obtained by a simple reading on the above 
mentioned curves. 

If X is regarded as a continuous variable, then ^i[X(pi), ^ 2 (^ 2 )] is a con¬ 
tinuous function of X, and, in fact, can be represented by a single cuiwe similar 
to those appearing in section 6. Moreover, as is apparent from (3.8), repeated 
differentiation of Ei[X{p^^ ^(^ 2 )] yields continuous derivatives. Consequently, 

t 

Ei[X(pi), k 2 {p^] can be expanded in Taylor series about o, where a = ^ viXi . 
This procedure gives 

(4.9) E{EAX(]>i), ^(Ps)]) = i) IT, Z £i[a(pi), ^(ps)], 

J-0 J 1 

where E{[a{p^y hip^] represents the yth derivative of JS7i[X(pi), A; 2 (p 2 )] with 
respect to X evaluated at a. Interchanging the order of summation one ob¬ 
tains 

j-o 3 1 <-i 


(4.10) 
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The final result then becomes 

(4.11) E(EtlX(p^, h(p^]) = i: 

J-0 J I 

where m/ Is the jth, moment of X about its mean, a. Thus to a first approxima* 
tion 

(4.12) EmXipi), h(p2)]) EiHpO, hiP2)]. 

It is of interest to note that if jE7i[X(pi)> fe(p 2 )] is linear in X then (4.12) is an 
exact expression since all derivatives except the first are zero. Furthermore, 
if Ei[X(j)i)j k 2 {p 2 )] is of the second degree in X, then only the second non-zero 
term on the right hand side of (4.11) needs to be added to (4.12) in order to 
make it exact. The former of these is the relation which gave an exact solution 
in 4.2. 

It is important to realize that this analysis for E(EilX{pi), h{p 2 )]) can be 
immediately applied to E(E 2 [X(pi), fe(p 2 )l). For, by the use of (3.14) and 
(4.8), one obtains 

(4.13) E(E,[X(pd, fci(P2)]) = r + 7 - S(B,[X(pO, fct(pO]). 

Pi P2 

The same analysis can be applied to Fh,i[ ] and the general result obtained 
that 

(4.14) £7(F,.i[X(pi), UP2)]) FhMPi). hM 

This immediately allows one to approximate the variance in the obvious manner. 

It is of interest to consider briefly the situation when both ki and fe are re¬ 
placed by random variables. Let ki be replaced by Xi taking on values Zu , 
Xi 2 , • • • , zit with probabilities , • • • , tth and fe be replaced by X 2 

taking on values Z 21 , Z 22 , • • • , X 29 with probabilities 7r2i, 7r22, ■ • • , ^ 2 #. Then 

(4.15) EiEi[Xi(j)i), Xiip^d]) = 22 irii^2iEi[xu(.Pi), Xsj(pj)l, 

t.y 

where i = 1, 2, • • • , < rndj = 1, 2, • • • , s. Again applying Taylor series 
and expanding about a = 2 iruXu and 5 = 22 *■»;%/ > the result is obtained 

X i 

that 

(4.16) EmXi(p,), X2(ps)]) = Hiu M2., 

where Ei’la(pi), i>(p 2 )] is the wth partial derivative with respect to Xi and 
the J>th partial derivative mth respect to X 2 of Si[Xi(pi), X 2 (p 2 )] evaluated 
at Xi = a, Xt = b. This gives the approximate formula 

E{Ei[Xi(pi), Xtipi)]) EdaipO, b(pi)]. 


(4.17) 
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4.4. Mean and variance of the number of trials required to obtain either (at 
least) ki balls in the first box, or (at least) fe balls in the second box, • • • , or (and 
at least) kn balls in the nth box. In accordance with previous notation the mean 
number of trials required is given by Eilkiipi), ^(^ 2 ), * * • , knipn)]- The exact 
value of this quantity can be written down and it w’ould be a complicated multi¬ 
nomial expression. The evaluation of such an expression would be extremely 
difficult, if not impossible, especially for large values of ki, k 2 ,•••, kn • In 
order to obtain an approximation to Ei[ ], repeated applications of (4.12) can 
be made and the resulting expression can be evaluated by means of the curves 
in section 6. 

For convenience, consider Ei[ki(pi), k^ip^, kz{p^, ki{p^]. The general 
result will then be apparent. Assume that the first three boxes form a single 
unit with probability (pi + p 2 + Pa). Then the number of balls required to 
obtain either ki in the first, in the second or kz in the third, if all balls are going 
in these three boxes, is a random variable X. Consequently, 

r4.18) Ei[ki{pi), • • • , A;4(p4)] = E{Ei[X{pi + pz + ps), A;4(p4)]). 

Applying (4.12), 


Eilkiipi), ••• , A-4(p4)] 


El El I^Ai + pg) > ^2 4. + P3)» + p2 + Pz) 


(Pi + P 2 + Pa), h(j)A) 


Applying (4.12) once again the final approximation is 


Ei[ki(pi), ••• , A'4(P4)] 




Pi + ^ 

Pi + P2 + PsJ 


KPI + P2 + PtJ 


(Pl + P2 + 


Pi), AriCp.) . 


Expression (4.20) can be translated into a course of procedure. One considers 
the first two boxes and computes 

It is then assiuned that Oi is a new number associated with a box with probability 
(pi + Pi) and 




Pa > 
+ Pa + PtJ 
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Repeating this procedure again, one computes <h = EiMpi + P 2 + Pa), AjiCpO], 
and by (4.20) this is approximately equal to j^i[A;i(pi), • • • , hipi)]. This method 
of computation is seen to be completely general and one can apply it to any num¬ 
ber of boxes. Each step consists of computing Ei[ ] for two boxes and con¬ 
sequently can be carried out with the curves of section 6. It is evident that 
the order in which the boxes are taken may have an important effect on the size 
of the error involved in using this step-by-step procedure. This problem will 
be considered in section 5. 

It is of interest to note that one can also obtain another approximation for 
Ei[ki(pi)j fe(p 2 ), A^(ps), h(pi)]. Suppose that the first two boxes are con¬ 
sidered as one unit and the second two boxes as another unit. Then the num¬ 
ber of balls which must fall in the first two boxes in order to obtain either ki in 
the first box or k 2 in the second is a random variable Xi . Similarly a random 
variable X 2 can be associated with the last two boxes. Accordingly 

(4.21) ^i[A;i(pi), * • • , hip^)] = E(Ei[Xi{pi + pa), Xiip^ -f P 4 )]). 


By use of (4.17), (4.21) can be written as 

■ ■ ■,hWl - E, [s.[i, (^-^), h (^^)] (P. + 


(4.22) 


^ (^.)' *• (^+ 7 .)] <*' + ''•]■ 


This same analysis applies directly to the factorial moments. In particular 


^’2.i[A:i(pi), • • • , ki(p^\ ^ Ft,i 


(4.23) 


[ ^' [ (pT^.) • (pTtt;)] ■ 

(p.+p;+7.)] (p-+p-+p-*. ‘-‘Wi- 


From (4.20) and (4.23) an approximate value for v?[A;i(pi), A; 2 (p 2 ), hips)^ k^ipi)] 
can be obtained. This procedure is also perfectly general and so an estimate 
of o-?[ ] can be obtained for any number of boxes. 

This same method can be immediately applied to the approximation of 
En[kiipi)y • • • , knipn)]- One simply considers the boxes two at a time, comput¬ 
ing E 2 [ ] at each stage instead of Ei[ ]. 

4.5. Solution for Eslk""] and Et[ki'~\ fe]. When s is different from 1 or n, 
the complexities of the problem force one into the consideration of only the 
quantities given in the title of this subsection. The corresponding problem 
for three boxes, namely ^ 2 [A:i(pi), fe(p 2 ), ^s(P8)], has been treated for general 
ki and pi by McCarthy [5]. However, the resulting expression is so complicated 
that it will not be given here. 

The process to be used consists of reducing the subscript s by a series of steps 
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until the subscript 2 is reached. This expression can then be evaluated by the 
use of the curves or by simple computation. For the sake of convenience, the 
case Ez[k^] \vill be considered in detail. It will then be possible to write down the 
expression for general s and n. 

As a starting point, look upon the first three boxes as a single unit. Then 
there is a definite probability ir< that one of these boxes will have k balls in it for 
the first time on the a;<th throw into these three boxes and that the other two 
boxes of the unit will each have less than k balls. Then if one of the other of 
the three boxes has u balls {u < k) the third box will have {xi — k — u) balls, 
(Xi — k — u < k). Meanwhile the fourth box will also have been receiving balls, 
and the number in it at this time will be denoted by j, (j = 0, 1, 2, • • • , oo). 
For each Xi there is a probability associated with u, namely P(u | Xi), and another 
probability associated with j, P(j | Xi), For the moment, consider that box 
1 has received k balls, box 2 the (xi — k — u) balls, box 3 the u balls and box 4 
the j balls. This numbering is of course immaterial since the situation is sym¬ 
metric with respect to the first three boxes. 

Now if j > /c, either {2k + u — Xi) balls will be required in the second box or 
{k — u) balls in the third box in order to obtain three properly occupied boxes. 
On the other hand, if j < k, the specified number will be required in any two of 
boxes two, three and four. Consequently, with this conditional description of 
the situation, the required number of balls necessary to obtain three out of the 
four boxes occupied in the proper manner is 


(4.24) Xi + j + E2[{2k + u - a;,), {k - w), {k - j)l 


where (^ — j) will be taken as zero if j is greater than or equal to k. From this 
description, it is evident tl^at the desired mean value may be obtained by sum¬ 
ming (4.24) over all possible values of Xi , j and w. Therefore 


(4.25) 


E^[k*] = Zir4x.+ ZF0-|a-.) 

t* L j-o 


I Xi)Et[{2k + M - Xi), (k -u),k- i)]^|. 


It is to be noticed that the probabilities inside the Ei[ ] in (4.24) and (4.25) 
do not add to one but only to 3/4. This can be eaaly remedied by the applica¬ 
tion of a formula similar to (4.6) and the result is obtained that 


E,[k*] 

(4.26) 


£ (xi + 53 P(J 1 Xi) 

i ( 1-0 

-f- 4/3 2 P(u I Xi)Et[(2k -t- M — a!j), {k — u), (k 



where each probability inside E 2 I ] is now 1/3. 
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By simple condderations 

(x{ A?)! 

27) = _, 

2 . -TT -''* (4r («•'"*■“ 


(J)“(i)' 


u — w)I 

where iz and (x< — — w) are both less than fc, and 


(ir(ir 




From (4.27) and (4.28) 
(4.29) 


Z jPU I Xi) = Xi/3, 


52 wP(u 1 a;,) 


— k 

“T” 


(4.25) can be written as 

Ei[k*\ = Z »r<Xi + 52 HjPU I Xi) 

i i i 

(4.31) 4 ^ ^ 

+ I Z «■< Z PQ I Xi) z P(.u I Xi)Ei[{2k + u- Xi), {k - u), {k - ;)]• 

0*2 U 

Finally, making use of (4.29), (4.30), the definition of Zi and tt,- and the procedure 
of replacing random variables inside an E 2 [ ] by their mean values, 

(4^2) FalA:*] , (ik - , (k - ^)]}, 

and this in turn can be written as 

(4.33) E,[k*\ ^ I |Pi(fc‘] + Pa ^(ik - (k - ^)]}. 

This method of analysis which has just been applied to Ez[k*] can be used 
equally well for Etlk""]. Here one simply considers the first (n — 1) boxes and 
proceeds as above. The final result is immediately apparent, namely that 

It will be noticed that in reducing (4.34) further it will be necessary to conader 
eYproaginna of the form E,[kx~\ fe]. However, it will be seen from the forgoing 
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analysis that no use was made of the fact that the integers attached to the first 
(n — 1) boxes were the same. Accordingly, 


(4.35) 


E.[kr^, h\ 


■ {fill 

n — 1 


[*?-*] + 


Ea—i 



ki 






Now, by the use of (4.34) and (4.35), it is possible to reduce 5 as much as may be 
desired. 


5. Some considerations concerning the error of the approximations. 

5.1. Preliminary remarks. This discussion of the errors of the approximations 
given in the preceding sections has been left until now so that a broad perspec¬ 
tive might be gained, and the errors seen in relationship to one another. Such 
an arrangement is advantageous in this instance since both the analytical and 
computational results bearing on the subject are scanty, and consequently, 
any intelligent leads which their inter-relationships can give are most helpful. 

The difficulty involved in obtaining exact values for the various quantities 
considered in this paper has been pointed out quite frequently, and the approxi¬ 
mations have been devised to overcome this very difficulty. The same com¬ 
plexity which prevents the computation of many exact values also prevents any 
effective analytic approach to the problem of evaluating the errors. For these 
reasons the author has been unable to carry through any general analytic treat¬ 
ment of the errors of the approximations. However, because the intelligent use 
of approximations requires some knowledge of their accuracy, certain isolated 
cases have been investigated by a combination of computational, graphical and 
analytic methods. These investigations are detailed in the remainder of this 
section, and conjectures concerning the general behavior of the errors are made 
whenever possible. As has been stated earlier, no consideration will be given 
to the approximation formulae for the variance. 

5.2. Errors of the approximations for Ei[ki{pf), • • • , knipn)] and 

En[ki{pi) y * • * , Ajn(Pn)]- 
Taking n equal to 3, we have from (4.11) that 

I Ei\ki(pi)y h{jh)y hip^)] - Ei[a(pi + pa), h{pz)] \ 

• Max|£![Xfp,+ /)*), Wll, 

where Max | E\[X(pi + pt), A:j(p»)] | is the maximum absolute value of the 
second derivative of Ei[X(pi + jja), Ar8(p8)l with respect to X, and a is equal to 
Ei[ki(pi/{pi -b Pa)), fe(p 2 APi + Pa))]- Now an examination of the curves 
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given in section 6 indicates that, for fixed ps and h , the maximum curvature of 
Ei[X(pi + P 2 ), kz(pz)]i considered as a function of X, is a monotone decreasing 
function of kz. Since this curvature is negative, this geometric observation is 
equivalent to 


Max I E\[X{pi + P 2 ), (Jkz + 1)(P8)] I 

(5.2) 

< Max I E\[X{p, + P 2 ), kz{pz)] I, 

although it is not necessarily true that 

I E\[Xi,{pi + Pa), (/ca + DCpa)] 1 < 1 E\[xi{,pi + P 2 ), fcaCps)] | ■ 

Moreover, 


(5.3) E.Mpd, k,{p 2 ), kzipz)] < E,Mp,), fe(p 2 ), (^•3 + 1)(P3)]. 

From (5.1), (5.2) and (5.3) one readily obtains that the absolute value of the 
percentage error of the approximation to Ei{ki{p^y fe(p 2 ), kz(pz)] is bounded‘by 
a function, say Ui[ki{pi), kzipz), kz(pz)], which is a monotone decreasing function 
of kz as kz increases. It should be noticed that the results of 4.2 have already 
shown not only that this upper bound for the percentage error approaches zero 
as kz becomes infinite, but also that the absolute difference between the true and 
approximate values approach zero as kz becomes infinite. 

Computation of Ui[ki(pi), h{p^, kz{p^] is very time consuming because of the 
difficulty in obtaining Max | E\[X{pi + P 2 ), A; 3 (p 3 )] I , and because the direct 
computation of Ei[ki(pi), fe(p 2 ), kzipz)] is laborious when any of A:i, fe and kz 
are much larger than 2 or 3. In order to surmount these difficulties and still 
give some indication of the behavior of Ui[ki{pi), hip^), A;3(p3)], the following 
expedients were adopted: 

1 . The values of A;i, fe and kz were each fixed at 5, 

2 . Max I E\[X{px + P 2 ), A; 3 (p 8 )] | was obtained by graphical means, namely 
drawing the slopes of the appropriate curve in section 6, graphing these slopes 
and then taking off the maximum slopes of these curves. 

5. Ei[ki{pi)y kzipz), kzipz)] was replaced by its approximation, 

Ei[a{pi + P 2 ), kz{pz)]y 

in the computation of the percentage error. This new bound will be denoted 
by Utlkiipi), kzipz), Wps)]. 

4 , Carefully chosen values of Ui[ki(pi)y k 2 {p 2 )t kzipz)] were plotted on trian¬ 
gular coordinates, and contour lines interpolated and extrapolated to cover in 
large part the range of pi, p 2 and pz . 

The use of the third of the above listed assumptions is no detriment to the 
usefulness of the results since 


Eia[] - Ell] __ Eia[] ^ Ui[ki(pi)yhijH)y kz(pz)] 

^7i[] , Elal] - Ei[\ ““ 100 - Utlkiipi), kziPiX h{pz)]' 

Eial] 
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where Eia[ ] = Ei[a{pi + P 2 ), A;*(p8)] and Ei[ ] = Ei[ki{pi), A:*(p 2 ), A^aCps)]. 
Since Ui[ ] is a monotone decrease function of ks , this new bound on the per¬ 
centage error is also monotone decreasing for increasing kz . Absolute values 
were not required in this derivation since Eia[ ] is always greater than or equal 
to Ei[ ], as is apparent from (5.1) and an examination of the curves of section 
6. The contours of^f/r[5(pi), 5(p2), 6(ps)] are shown in Fig. 1. The interpreta¬ 
tion of this figure is very straightforward. For example, for pa ^ .5, the value 



* 7.S3t 5.0J5 2.055 

Fig. 1. Contours op C/*[6(pi), 5(p2), 5(pi)l Considered as a Function 

OP Pi, p2 AND Pi 


of l/f[5(pi), 5 (p2), 5(p 3)] is less than 5.0%. Making use of the definition of 
Ui[ ], and especially its monotone characteristic, one can then say: the ap¬ 
proximation for £^i[5(pi), 5 (p 2), AJaCpa)], where kz ^ 5, ps ^ .50 is in error by not 
more than 5.3%. More 9 ver, as has been already observed Ei[aipi + P2), 
A^(P8)] is always greater than or equal to JE^i[A;i(pi), A^(p 2 ), A^sCps)]. 

It will be noticed from Fig. 1 that Ui[ ] is increasing steadily as ps approaches 
1. It has been demonstrated by McCarthy [5] that this behavior of the upper 
bound does not mean that the percentage error itself becomes larger as pa ap- 
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proaches 1 . As a matter of fact, for fixed ki , k 2 and h , the percentage error 
approaches zero as ps approaches 1 . However, this demonstration does not 
furnish any reasonable bounds with which to fill in the lower left hand comer of 
Fig. 1 . This fact is not as serious as it may at first seem because there is nothing 
to prevent one from reordering the boxes. For example, consider Fi[5(.2), 
5(.2), 5(. 6 )]. From Fig. 1 , the error of the approximation for this quantity, 
namely Ei[Ei[6i.5)y 5(.5)](.4), 5(.6)], is not more than approximately 

7.5/(100 - 7.5) =- 8 . 1 %. 

On the other hand this same figure shows that Fi[Fi[5(.25), 5(.75)](.80), 5(.20)], 
which is also an approximation to Ei[5(.2), 5(.2), 5(.6)], is in error by not more 
than approximately . 8 %. (consequently one would choose the second ordering. 

The procedure which has been used to obtain an upper bound on the percent¬ 
age error of the approximation to Ei[ki(pi)f fo(p 2 ), feCpa)], h and fixed and 
fe greater than or equal to that integer at which the bound is evaluated, can also 
be applied to Ezlkiipi)^ fe(p 2 ), faCpa)]. All the assumptions remain the same 
and in this case the bounds corresponding to Ui[ ] and Utl ] are denoted by 
f/af ] and Ut[ ]• As in the case of f/d ], we have 

j j j 

EA] - E^[] ^ Enl] < Ut[k,(pd, kAjh), hip>)] 

EA] , , Ball - E«,[] - 100 

B36[j 

Here the approximation, F 2 [ 6 (pi + P 2 ), hipi)], is always less than or equal to 
the exact value, E^lhipi), fe(p 2 ), A^aCpa)]. The contours of (/a [5(pi), ^(pa), fiCpa)] 
are shown in Fig. 2. In using (/*[5(pi), 5 (p 2 ), 5(p8)] it is sometimes advan¬ 
tageous to reorder the boxes. For example, consider £' 3 [ 5 (. 2 ), 5(.2), 5(.6)]. 
Fig. 2 shows that, as an approximation, j& 2 [£^ 2 [ 5 (. 5 ), 5(.5)](.4), 5(.6)] is in error 
by not more than approximately 9%. However, F2[£?2[5(.25), 5(.75)](.80), 
5C20)], which is also an approximation for F8[5(.2), 5(.2), 5(.6)], is in error by 
not more than about 7%. There is a gain here, but it is not as great as the cor¬ 
responding situation for jE7i[5(.2), 5(.2), 5(.6)]. 

As has already been stated, one may minimize the error by correctly choosing 
the two boxes which are to be combined first. Some discussion will be given 
here of a procedure for choosing these two boxes. Of course an experimental 
scheme may be used which makes use of the fact that the approximation to 
Ei[ki(pi), k 2 {p 2 ), kaips)] is always an overestimate. In other words, that grouping 
is us^ which gives rise to the smallest value of the approximation. However, 
this can be replaced by a few preliminary computations. 

As can be seen from (5.1), the error of the approximation depends upon two 
quantities, namely the variance of the two box situation obtained by combining; 
two of the boxes, and the maximum value of the second derivative of the curve 
representing the function Ei[Xipi + P 2 ), A; 8 (p 8 )] over the proper range of X 
values. The error will be zero of Ei[X(pi + P 2 ), hipn)] is either a constant or 
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linear in X over the range of X values in which one is interested, that is ki < 
X < — 1, Ati ^ fe . If this is not possible, then one wishes to make it 

as near so as possible, subject to the restriction that 

O’lfAjiCpi/Cpi + P 2 )), fe(p 2 /(pi + Ps))l 
is not unnecessarily large. 



■* 16*05J 10,056 

Fig. 2. Contours of U *[ b {' p\)j 5(p2), 6(^3)] Considered as a Function 
OF pi, p2 and P3 


An indication of the relationship between the boxes for both linearity and con¬ 
tribution to variance can be obtained from expressions (3.10) and (3.11). Thus 
for each box one computes h/pi and A;*'(l — p,) /p\. Then in order to most nearly 
achieve linearity one orders the boxes in accordance with the increasing order of 
ki/pi and combines them in that order. If there is a tie between two or more 
boxes with respect to the‘A;,/pi ordering, then one orders these ‘‘tied” boxes in 
accordance with increasing A;<(1 — p<)/p<. 

Some computations have been carried out to illustrate these points and they 
are given in Table 1. The notation ((2,4), 6) means that one first combines the 
boxes with integers 2 and 4, and then combines this result with the box with 
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associated integer 6. All values in this table were obtained by direct computa¬ 
tion. No use of the curves was made. 

In these three situations, one obtains the values given in Table 2. 

Thus in the first case there is nothing to choose with respect to , but 

A;*(l ~ indicates the ordering ((6, 4), 2). Actually the percentage error 

in this instance is 1.0 as compared with 1.7 and 2.4 for the other two orderings. 
In case two, ki/pi indicates the ordering ((2, 6), 4). Although this does not 
turn out to be the best ordering, Table 1 shows that the ordering in this instance 
makes little difference. In the last case, the indicated ordering is ((2, 4), 6) 
and the percentage error for this is zero, as opposed to 1.3 and 1.6. Since at 
any stage in the operation of combining boxes two at a time (4.13) holds, the 

TABLE 1 


Effect of Order of Combination on Error of Approximation 


Vi 

V. 

Pi 

H 

k2 

kt 

Ei[ki{pi), ktipt), ktipz)] 

% Error of Approximation 

Order of Combination 
((2, 4), 6) ((2, 6), 4) ((4, 6), 2) 

2 

4 

6 

6.96 

+ 1.7 

+2.4 

+1.0 

4 

6 

2 

3.92 




6 

4 

2 

3.77 


+1.3 

+1.6 


TABLE 2 


Pi 

1/6 

1/3 

1/2 

1/6 

1/3 

1/2 

1/6 

1/3 

1/2 

ki 

2 

4 

6 

4 

6 

2 

6 

4 

2 

ki/Pi 

12 

12 

12 

24 

18 

4 

36 

12 

4 

— Vi)/p\ 

60 

24 

12 

120 

36 

4 

180 

24 

4 


above procedure will also give the minimum error for the approximation to 
Eziklipi), kzipz), A; 8 (p 3 )]. Moreover, the approximation for this quantity is always 
an underestimate of the true value, and therefore that ordering should he taken which 
gives the greatest value for the approximation. 

When the error of the approximation to Ei[ki{p^, • • • , A:«(p„)] and 

Evfk\{jp^, • • * , kn{Prd]t 

for n greater than three, is considered, it is immediately obvious that the general 
considerations already given in this section still apply. In addition to these 
considerations, there is the difficulty that errors may cumulate. However, the 
results already quoted for three boxes, in conjunction with those which are to 
be given in 5.3, indicate that this cumulation is not serious. There are two 
factors which eventually prevent (i.e. as more and more boxes are considered) 
this percentage error from becoming unduly large, and, in fact, make it approach 
zero. These are: 

L The value of pz will, in most instances, be decreasing as more and more 
boxes are considered (see Fig. 1), and 
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2. The true value is usually becoming larger and larger as more and more boxes 
are considered. 

In order to minimize the error, the following precautions should be taken: 

1. At each stage in the computation, try to avoid, as much as possible, making 
readings where Ei[X{j)[ + p 2 ), A^aCpa)] is curving sharply. If all readings are 
made where the curves are nearly linear, the percentage error will be very close 
to zero. On the other hand, if many readings must be made where the slopes 
of the curves are changing most sharply, larger errors must be expected. 

2. Use that ordering of the boxes which provides the minimum value for the 
approximation to ] or the maximum value for the approximation to ]• 

3. In order to approximate the ordering which (2) would give, compute 
kt/px and A:,(l — pt)/pi at each stage at which two boxes are to be combined 
and use the rules of procedure already given for three boxes. 

5.3. Error of the approximation for Eg[kf]. Repeated applications of the re¬ 
duction formulae (4.34) and (4.35) allow one to evaluate E»[k^] by means of the 
solution for the two box case, or more explicitly, by means of the curves given in 
section 6. Here the error of this approximation will be discussed primarily from 
a computational point of view. 

Et[V] can be treated in detail since it is possible to obtain exact values for this 
expression by means of (4.1). This has been done by McCarthy [5], but the 
details will not be repeated here because of lack of space. The results simply 
add more credence to the conjectures which will soon be made. 

When k is taken to be larger than one, the difficulty arises that it is almost im¬ 
possible to compute the exact value of Et,[k^] in a large number of cases. Con¬ 
sequently it w^as necessary to devise an experimental model to estimate these 
exact values so that the amount of error would be known within bounds. A 
set of 10,000 punched cards^ w^as obtained on which w^ere recorded 100,000 
random numbers drawn from a rectangular distribution. Thus if the cards are 
ordered on a particular set of columns, and one reads off the digits 0-9 on another 
specified column, one card at a time, it is equivalent to using a table of random 
numbers such as those prepared by Tippett [7]. By the use of these cards, it 
was possible to run off on an IBM Tabulator any desired number of experiments 
in order to obtain an experimental distribution from which to calculate an es¬ 
timate of and the variance of this estimate. For example, in determining 
an estimate of ^i[2®] one hundred experimental trials were made, as described 
above, with the following results: 


Number of Trials 
Required 

2 

3 

' 4 

5 

6 


Frequency 

23 

32 

31 

11 

3 


^ These punched cards were prepared at the Mayo Clinic, Rochester, Minn., under the 
direction of Doctor Joseph Berkson. 
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From this distribution the estimate of ^i[2®] is 3.39, with a variance computed 
from the distribution of .011. The 95% symmetric confidence limits for the 
mean, computed from the Student ^distribution, are 3.17 and 3.61. Such 
estimates will be used in the remainder of this section. It should be pointed 
out that in order to prevent a prohibitive amount of machine time, it was 


TABLE 3 
Percentage Errors for 


8 k 

n 3 

4 

5 

1 1 

— 

_ 

_ 

2 

+ .7 

+ 2.2 

- .3 +13.6 

5 

+ 1.1 

- 3.1 +5.7 

+ .6 +10.7 

10 



- 2.9 + 5.1 

2 1 

- 5.6 

- .4 

+ 1.3 

2 

- 4.6 

- 4.4 +4.4 

+ .6 +10.4 

5 

-4.6 -fl.7 

+ 3.0 +9.3 

+ 7.9 +14.8 

10 

-3.7 +2.1 

- .3 +5.5 

+ 4.3 +10.7 

15 


+ 1.0 +7.2 


20 

-2.5 +2.4 



3 1 

-18.2 

-12.7 

- 3.1 

2 

- 6.3 

-16.5 -7.3 

- 2.9 + 6.0 

5 

-9.7 -2.2 

-10.7 -5.5 

+ .8 + 5.8 

10 



- 2.1 + 3.1 

4 1 


-12.0 

-15.6 

2 


-13.6 +6.1 

-11.6 - 3.9 

5 


-13.9 -7.2 

- 9.9 - 4.0 

10 


- 8.9 -2.6 

- 6.4 - 1.2 

5 1 



-8.8 

2 



-18.1 - 6.0 

5 



-12.5 - 5.6 

10 



- 8.9 - 2.9 


necessary to use many of the same runs to determine values of Ea\k^] for different 
values of s, k and n. This means that the errors are correlated to some slight 
extent, but it would be extremely difficult to determine how much. 

A summary of the computed percentage errors for various values of s, k and 
n is given in Table 3. In the instances where there are two entries, they are 
calculated on the basis of the 95% confidence limits for the experimental mean. 
These confidence limits are symmetric and weie determined by using the Student 
t-distribution. For k equal to 2 and 5 the distribution contained 100 trials, 
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while for k greater than 5, the distribution were made up of approximately 50 
trials. 

The computations given in this table show for various values of s, k and n, 
the percentage error of the approximation for In addition to showing 

the values of these percentage errors, the computations lead one to conjecture 
that 

1, For fixed s and k, there exists an no such that for n > no the absolute value 
of the percentage error of the approximation for £?.[/:"] is a monotone decreasing 
function for increasing n. It was shown by McCarthy [5] that this absolute 
value approaches zero as n approaches infinity for Fa[l"], and in fact, that the 
difference between the true and approximate values approaches zero. 

2, For fixed s and n, there exists a A^o such that for k > ko y the absolute value 
of the percentage error of the approximation for E^W] is a monotone decreasing 
function for increasing k. 

6. Computation. 

6.1. Curves to aid in the computation of Ei[ki{pi), fe(p 2 )]. In 3.1 it was shown 
that Ei[ki(pi), kiijh)] is equal to 

■ + 1, A*2) + ^ Ipiik2 + 1, Aji), 

Pi P2 

where /*(p, q) is the Incomplete Beta-Function as tabled by Karl Pearson [6]. 
There are three principal difficulties connected with the use of these tables as 
they apply to the approximations of this paper. These are: 

L The tables must be available, 

2. The tables give directly only values for integer or half-integer values oi 
ki and k 2 , and 

3. Since many different values of Ei[ki{pi)y kaijh)] are often required to obtain 
a single approximation, the computational burden would be very heavy. 

In order to surmount these difficulties, it seemed advisable to prepare curves 
giving the values of Ei[ki{pi), k 2 {p 2 )] for various values oi ki, k 2 , pi and pj. 
These curves would give values of ^i[ ] with sufficient accuracy for most prob 
lems not only for integer values of ki and k 2 , but for all values over the range 
considered. 

Such curves have been prepared by computing Ei[ki{pi)y k 2 ip 2 )] for integral 
values of Aji and k 2 (for fixed pi and P 2 ) and then joining these points with a 
smooth curve. A summary of the graphs prepared is as follows: 




*1 

h 


Pi 

pi 

Fig. 3 

1,2, • 

• • , 25, « 

1, 2, •• • 

,35 

.50 

.50 

Fig. 4 

1,2, • 

• * , 20, 00 

l, 2, • • • 

,35 

.40 

.60 

Fig. 6 

1,2, ■ 

• • , -15, 00 

1,2, • • 

,35 

.20 

.80 

Fig. 6 

1,2, • 

• • , 10, <30 

l, 2, • • • 

,15 

.80 

.20 

Fig. 7 

1,2, ■ 

• • , 7, 00 

1,2, ••• 

,15 

.60 

.40 

Fig. 8 

1,2, ■ 

• • , 8, 00 

1,2, ••• 

,15 

.50 

.50 

Fig. 9 

1,2, ■ 

• • , 6, 00 

1,2, ••• 

,15 

.40 

.60 

fig. 10 

1. 2,- 

■ • , 5, 00 

1.2, ••• 

,15 

.20 

.80 
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Figures 8, 9, and 10 are simply portions of figures 3, 4 and 5 drawn on an ex¬ 
panded scale in order to permit greater accuracy in reading the curves. Also 
figures 6 and 10 and figures 7 and 9 form pairs in that a member of one pair can 
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be obtained from the other member of the pair. Both members of the pair are 
given on the expanded scale in order to facilitate interpolation. Values of the 
mean for combinations of A;i and not given directly can usually be obtained 
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with sufficient accuracy with linear interpolation. Interpolation for pi and pt 
should be done graphically since in some instances linear interpolation would be 
extremely poor. 
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As an example, suppose one has two boxes with ki = 2, fe = 5, pi = .40 and 
P 2 = .60. Consulting Fig. 9, one goes along the horizontal axis to kz — 5. 



0 I 2 3 4 5 6 7 8 9 10 II 12 13 14 


FoUowing up the vertical line through this point to the curve h = 2, iFi[2(.40), 
6(.60)] is read as 4.25. The actually computed value to four decimals is 4.2224. 
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It is immediately e\ident that E 2 [Ki{'p^, can also be obtained from the 

curves since 

^2[A;i(pi), fcj(p2)] = {ki/pi) + (VP2) “ £7i[/ci(pi), fe(p2)]. 

6.2. Use of the curves to obtain exact values (t.e, subject only to the error of reading 
the curves) for Ei[ki(pi), ^sCps)]. Referring back to (4.8), one obtains 

that 

( 6 . 1 ) 


Ei[ki(pi)y hiv^y k^ips)] == JL v^EiMpi + P 2 ), A:8(p«)], 
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0 1 2 3 4 5 6 7 8 9 10 II 12 13 14 


where tt* is the probability that either k\ balls are obtained in the first box or k% 
balls are obtained in the second box on the Xi th throw for the first time, assuming 
balls can go only in boxes one and two. Xi takes on values 

fci, All + 1, • • • > ^ + — 1 

when ki < , Now tt, can be easily computed and Ei[xi{pi + P 2 ), hips)] 
can be obtained from the curves. The only difficulty in using this procedure 
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arises when the range of Xt is large. Then a large amount of computation is 
involved. 

In order to illustrate this computation, consider Ei[2(.l), 3(.l), 5(.8)]. Here 
Xx takes on the values 2, 3 and 4. We have x\ = 2 , tti = 2 /8; = 3, ira «= 3/8; 

and xz = 4, tts = 3/8. From Fig. 6 

J5;i[2(.2), 5(.8)] = 6.09 
^i[3(.2), 5(.8)] = 6.88 
^i[4(.2), 6(.8)] = 6.11. 
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0 I 2 3 4 5 6 7 8 9 10 Jl 12 13 14 

Consequently, j&i[2(.l), 3(.l), 5(.8)] is equal to 


(5.09)(2/8) + (5.88)(3/8) + (6.11)(3/8) = 5.77. 

Using computed values for Ei[xi{2), 5(.8)J, j&i[2(.l), 3(.l), 5(.8)] is equal to 
6.75. Thus the use of the curves has only led to an error of .3%. 
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6,3, Use of the curves in approximating ^i[fci(pi), • • • , Kipn)], 

EnlHpi), • • • , kniPn)] 

and In illustrating the application of the curves and the reduction 

formulae (4.34) and (4.35), one example will be worked through in detail. This 
example will provide illustrations of all the details involved in such problems. 
Consider ^4[5®]. Applying formula (4.34) 

(6.2) E4[5‘J 5/4 + £, 5 - , ^5 - ■^)]} • 

Consequently, the first step must be to compute Using the principles of 

4.4 

(6.3) ^i[5'] Eim5%50), 5(.25), 5(.25)]. 

From Fig. 8, Ei[6^] = 7.55. Therefore £7i[5^] is approximately equal to 
£;i[7.55(.50), 5(.25), 5(.25)]. 

Now applying the same principle again, 

(6.4) E\[5^] ^ EAEi[7.55{i), 5(J)](.75), 5(.25)]. 

By the use of figures 7, 8, 9 and 10, graphical interpolation may be applied to 
find that A\[7.55(3), 5(^)] is equal to 9.84. The approximation procedure now 
says that 

(6.5) Ei[5*] Zi;i[9.84(.75), 5(.25)]. 


Again applying the curves and using graphical interpolation for pi and p 2 , 
Ei[5*] 11.88. 


Substituting this value in (6.2), 

(6.6) Ei[5’'] - f {11.88 + £?3[2.71, 2.71, 2.71, 2.03]}. 

Now formula (4.35) must be applied to E3[2.7l, 2.71, 2.71, 2.03], i.e. 
£;3[2.71, 2.71, 2.71, 2^ 3] 


(6.7) 


i | ej [( 2 . 71)’1 + E, [^3 2.71 - , ^2.03 - j|. 


Ei[{2.71)^] can be evaluated by the same method used for Ei[5% This leads to 
the result 


(6.8) J5;8[2.71. 2.71, 2.71, 2.03] ^ {4.40 + £:2[1.86, 1.86, .56]}. 

Once more applying (4.35) 

E2[1.86, 1.86, .56] ^ 

|| e ,[(1.86)‘1 + £?,[(2-1.86 - £ j [(1.86)* l ), ^.66 - 


(6.9) 
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£^i[1.86, 1 . 86 ] is equal, by the curves, to 2.25. Therefore 

(6.10) ^*[ 1 . 86 , 1.86, . 66 ] J {2.26 + Ei[lA7, - . 66 ]}. 

However, since the convention is observed that a negative quantity is replaced 
by zero, 

( 6 . 11 ) Ei[lA7, - .56] = JE7i[1.47, 0] = 0 . 

Now working back through these various expressions, 

( 6 . 12 ) E4[5^] ^ f [11.88 + i [4.40 + J [2.25 + 0]]] = 27.81. 

From Table 2 it can be seen that the percentage errors for this approximation 
to Ei[5% corresponding to the 95% confidence limits for this quantity, are —4.0% 
and —9.9%. 

This example has illustrated most of the situations which will arise in the use 
of the approximations of this paper. 

6.4. Miscellaneous approximation formulae useful for computation. There 
exists a relatively simple approximation to Ei\ki{p^, ^^ 2 (^ 2 )], Pi + P 2 = 1 , when 
P 2 is near one. Using (3.8) and making some obvious simplifications, one ob¬ 
tains 

A;,W1 = ^ r 

P 2 P 2 {fCi — 1)!(«'2 — l)i Pi *'0 

Since pi is near zero, (1 — t) can be replaced by one, and the result is obtained 
that 


EAkAp,),W] 

V2 


1 Hi {kl + A’ 2 ) I 

(h+l)l(A'2- 1)I‘ 


An approximation to the Incomplete Beta-Function, given by Tukey and 
Scheff4 [8], may also prove useful at times. The expression, changed shghtly 
by those authors since publication, is 


h{n — r 4 - 1, r) — 1 


1 

2r(r) 



dx\ 


where 


2 

Xa 



r 

V6 


+ 2r. 


The right hand side of the first expression will be recognized as the x distribu¬ 
tion with 2r degrees of freedom. In the event that the tables of x are not ade¬ 
quate for the application of these expressions, the approximation of Wilson and 
Hilferty [10] should be used. This approximation states that (xV*')* where 
V is the number of degrees of freedom, is approximately normally distributed 
with mean 1 — 2/(9v) and variance 2/(9v), for large v. 
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THE DISTRIBUTION OF THE RANGE’ 

By E. J. Gumbel 
Brooklyn College, N. Y. 

1 . Summary. The asymptotic distribution of the range w for a large sample 
taken from an initial unlimited distribution possessing all moments is obtained 
by the convolution of the asymptotic distribution of the two extremes. Let a 
and u be the parameters of the distribution of the extremes for a symmetrical 
variate, and let R = a(w—2u) be the reduced range. Then its asymptotic 
probability and its asymptotic distribution \l/(R) may be expressed by the 
Hankel function of order one and zero. A table is given in the text. 

The asymptotic distribution g(w) of the range proper is obtained from ^(R) 
by the usual linear transformation. The initial distribution and the sample 
size influence the position and the shape of the distribution of the range in the 
same way as they influence the distribution of the largest value. If we take the 
parameters from the calculated means and standard deviations, the asymptotic 
distribution of the range gives a good fit to the calculated distributions for normal 
samples from size 6 onward. Consequently the distribution of the range for 
normal samples of any size larger than 6 may be obtained from the asymptotic 
distribution of the reduced range. 

The asymptotic probabilities and the asymptotic distributions of the mth 
range and of the range for asymmetrical distributions are obtained by the same 
method and lead to integrals which may be evaluated by numerical methods. 

2. Introduction. For any initial distribution, and any sample size n, the dis¬ 
tribution of the range may easily be written down in the form of an integral. 
However, for many given initial distributions the integration can be carried out— 
if at all—only for very small sample sizes, say n = 2 or n = 3, For larger 
samples, complicated numerical calculations have to be made, and there is no 
way of obtaining the distribution for n 1 observations from the distribution 
for n observations. 

Our object is to obtain the asymptotic distribution of the range. Nothing is 
supposed to be known about the initial distribution, except that it is of the ex¬ 
ponential type [9] which assures that it is unlimited in both directions, and pos¬ 
sesses all moments. It will be shown that this condition is sufficient for the 
existence of an asymptotic distribution of the range. 

With increasing samples sizes the distribution of the range may approach its 
asymptotic form in a quick*, or in a slow way. This behavior depends upon the 
nature of the initial distribution. Two examples for this approach will be 
shown. 


* Research done with the support of a grant from the Social Science Research Council. 
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3. The exact distribution of the range. Let ip{x) be any initial distribu¬ 
tion, $(.r) the probability of a value equal to, or less than, x. Then, for samples 
of size n, the joint distribution , Xn) of the smallest value Xi and the largest 
value Xn is 


(1) , Xn) — n{n — l)^(xi)(4>(xn) — ^(xi))” Vw. 

The distribution gn{wn) of the range Wn defined by 

(2) Xn^ Xi + Wn 

is obtained by integrating over all values Xi g x« whence 

(3) (Jniwn) = n(n - 1) f (^(x + wj - + Wn)ip{x) dx, 

J—00 


where the index 1 has been dropped. The probability GniWn) for the range to 
be equal to, or less than, Wn is obtained by integration of (3), whence, by re¬ 
versing the order of integration, 

f +OO 

/ (n — !)($(* + Wu) — di(x + w„) d^ix), 

00 ^0 


or, after integration, 

CrnCiOn) = n f (^(x + Wn) - #(x))"‘ ’ d#, 

JO 

a formula to which Prof. H. Hotelling has drawn my attention. The beauty of 
this formula is completely marred by the facts that, in general, wc cannot express 
4>(x + Wn) by $(x), and that the numerical integration is lengthy and tiresome. 

The problem of the range for the normal distribution was first raised twenty 
five years ago by L. von Bortkiewicz [1,2]. For 7i = 2 and n = 3 the distribu¬ 
tion of the normal range may be written down explicitly [12, 13). For larger 
noimal samples up to n = 20, E. S. Pearson [IGj and H. O. Hartley [10] have 
calculated numerical tables of the probability of the range. L. H. C. Tippett 
[20] has calculated the mean, the standard deviation, and the moment quotients 
for the range of the normal distribution up to n = 1000. He gave formulae for 
the moments in the form of integrals. Finally “Student'^ [18] reproduced the 
distribution of the range for small samples, n = 2, 3, 4, 5, 0, 10, by Pearson's 
type I, and gave a formula for large samples n = 20, 60, based on Pearson's type 
VI, a procedure which is purely empirical and, therefore, imsatisfactory for 
theoretical purposes. A good resum4 of the present knowledge about the 
range is given in Karl Pearson's Tables [17], 

All these studies are confined to the normal distribution and allow no conclu¬ 
sion about the asymptotic distribution of the range. According to Kendall [11] 
it is not known whether such forms exist and what they are. This question may 
at once be answered for a special case. If the distribution is limited to the left 
(or to the right), the asymptotic distribution of the range is equal to the asymp- 
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totic distribution of the largest (smallest) value. The asymptotic distribution 
of the range exists provided that an asymptotic distribution of the largest 
(smallest) value exists. For the exponential distribution, and for initial dis¬ 
tributions of the Pareto type, for example, the asymptotic distribution of the 
range is equal to the asymptotic distribution of the largest value. The asymp¬ 
totic distribution of the range for the rectangular distribution has been derived 
by A. G. Carlton [3]. 

4. The as3rmptotic distribution of the reduced range for a sjrmmetrical 
variate. Instead of the procedures mentioned in the last paragraph, let us 
consider a large sample. It is generally assumed that the smallest and the 
largest values are independent in that case. L. H. C. Tippett [20] has shown 
that the correlation between the extremes is negligible for the normal distribution 
and for sample sizes n ^ 200. In a previous note [9] it has been shown that 
independence holds for large samples and for initial distributions of the ex¬ 
ponential type unlimited in both directions and possessing all moments. Then 
the joint distribution (1) splits into the product of the asymptotic distribution 
/i(xi) of the smallest value X\ and the asymptotic distribution/n(a;n) of the largest 
value Xn 

( 4 ) tO(Xi , Xn) = fliXi) ’fniXn). 

If, furthermore, the initial distribution is symmetrical about zero, the two 
asymptotic distributions are 

(5) fiiXi) = a exp[a{xi + u) - ; /„(x„) = aexp[ - a{xn - u) - 

These asymptotic distributions and the corresponding probabilities are traced, 
in a reduced scale, on Graphs (1) and (2). 

Since the two parameters u and a will exist also in the asymptotic distribution 
of the range, their nature must briefly be explained. The value u is defined as 
the solution of 

(6) Hu) = 1 - i. 

n 

Since 

(6') n(l - $(w)) = 1, 

the largest value u may be called the expected largest value. It differs, of course, 
from the mean of the largest value. It has been shown [6] that u increases as 
a function of the logarithm of n, the function depending upon the initial dis¬ 
tribution. 

Criteria for the approach of the distribution of the largest value toward its 
as 3 anptotic form have been given by R. A. Fisher and L. H. C. Tippett [4]. 
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For our purpose it is sufficient to consider whether n is so large that u is very 
near to the most probable largest value Xn obtained from 


(7) 

If 


n — 1 




*p{Xn) 


Xn ! 


holds with sufficient approximation, 2u may be interpreted as the range of the 
modes for an initial symmetrical distribution. 

The parameter a defined by 


( 8 ) 


__ 

“ 1 — ^(w) 


also is a function of n. Three cases have to be distinguished: In the first case, a 
is a constant, or converges with n toward a constant different from zero. In the 
second (and third) case, a increases with n without limit (decreases with n 
toward zero). The three cases correspond to three classes of initial distributions 
of the exponential type. The function a is related to the as 3 maLptotic standard 
error of the largest, and of the smallest value by 


(9) 


2 2 2 2 ^ 
a (Tn = a ai = -r 
0 


If a increases (decreases) with n, or is independent of n, the standard error of 
the largest value decreases (increases) with the sample size, or is independent 
of it. This behavior has nothing to do with the fact that the standard error of 
the mean decreases, of course, with an increasing number of samples. 

The determination of the constants u and a from equations (6), (7), (8) is 
based on the knowledge of the initial distribution and the sample size n from 
which we take the largest observation. This method cannot be used in many 
practical applications: 1) It may happen that the initial distribution, or the 
parameters it contains, are unknown. Therefore the parameters of the largest 
value cannot be obtained from it. 2) The initial distribution might be known, 
but the number of observations is insufficient to warrant this procedure, because 
the most probable largest value Xn differs from the expected value u. In these 
cases the parameters u and a have to be estimated from the observed distribution 
of the largest value alone. A similar procedure will be used for the range in 
paragraph 7. 

From (4) and (5) the joint asymptotic distribution w) of the smallest 
value xi and the range w becomes 

w) - «*exp|-«(«) - 2«) - 

The asymptotic distribution g^w) of the range alone is, dropping the index 1, 
(40 g(w) = dx. 
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This distribution contains the two parameters a and u existing in the asymptotic 
distribution of the largest value. To eliminate the two parameters, a reduced 
range R is introduced by 

(10) R = a(w — 2u). 

The range w is & positive variate unlimited toward the right. The reduced 
range R is also unlimited toward the right yet limited toward the left by 

(10') R ^ -2otu. 

The reduced range is not related to one of the averages of the range. It is the 
range minus the range of the modes divided by a factor which is proportional to 
the standard error of the extreme value. The distribution \I/(R) of the reduced 
range /?, and the distribution g(w) of the range w are related by 

( 11 ) = ^-g(w), 

a 

subject to restriction (10'), whereas the probability <lf{R) of the reduced range to 
• be equal to, or less than R is equal to the corresponding expression G{w) for the 
range proper 

( 11 ') ^(R) = G(w). 

For the integration in (4') we put 

a{x w — u) = —y 


whence, from (10), 


a{x + u) = — y — R, 

The asymptotic distribution of the reduced range becomes 

( 12 ) il/(R) = [ exp[—e^ dy 

J—ee 

and the asymptotic probability ^(i?) of the range is 

(13) ^(R) = f exp[y - e* — e"""®] dy 


an expression which may easily be verified by differentiation. 

The asymptotic formulas (12) and (13) hold for any initial symmetrical dis¬ 
tribution of the exponential type, for example, for the normal and the logistic 
distribution (see par. 7). The mean reduced range R and the higher moments 
of the reduced range are easily obtained from the mean W, the variance al ,, and 
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the invariants X, of order v of the range proper w given in a previous paper [ 8 ], 


They are 



(14) 

= 2w + ~ ; ai 

2 


a 

~ 3a2 

(15) 

^ 2(v- 1)1^1 

' a' k’ ’ 

I' ^ 2 


where 7 stands for Euler's constant. 

Consequently the mean R, the variance <t\ and the invariants X, of the reduced 
range are 

(16) R = 27 ; = X, = 2 (»' - 1)1 E 1 • V^2 

o Jb-i 

Equation (14) leads to an interpretation of the reduction ( 10 ) which may be 
written 


R = a(w — w) 2y 


or 

(140 


R = 


3 o’to 


i 


Thus the transformation (10) is a linear function of the standard transformation 
(w — usual in statistics. 


5. The probability of the range as a Bessel function. The integrals 
(12) and (13) may be evaluated by numerical procedures, since tables of the 
function exp(—e~*') are easily calculated. However, it turned out to be simpler 
to relate these integrals to the solution of a differential equation. The deriv¬ 
ative ^'(i?) of the distribution ( 12 ) is 

s= —^(/g) -f f exp[— 2/ — 


The integral is equal to the probability ^(R) since the transformation 

y + R = -z 


leads to 



e dy = f exp[z — e * * ~ e*] da; 

J—OO 


Consequently the probability '^{R) is subject to the differential equation 

(17) ^ = 0. 
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The mode of the reduced range is a fixed valueJjS^Jsuch that 

(18) = e^h(R). 

Mr. W. Wasow (Swarthmore College) has drawn my attention to the fact that 
the probability ^(iB) of the range can be expressed in terms of a Bessel function.* 
To obtain this simplification of the differential equation we introduce a new 
positive variable z by 

(19) 2 = 
and a new function U by 

(20) ^ = U-z. 

The boundary conditions are 

(21) 2 = 0, ^ = 1; 2=oo; St^ = 0. 

The first derivative becomes, from (19) 

_2 

, dR 2 dz 

whence, from (20) 



The second derivative becomes, by the same procedure 

^ ^ d (__zU _ z^ d_U\ 

dR^ 2 d2\ 2 2 dz)' 

The second member may be written 

2V2 2 dz’^ 2 dz^) 4 4 d2 4 d22 * 

Thus the differential equation (17) is now 

^dV _ _ f! rr ^ Q 

4d22'^4d2 2d2’^4 2 4^ 

Multiplication by 42”^ leads to 

(210 + 

This is one of the classical Bessel differential equations of order 1. In the nota¬ 
tion used by the British Tabbies [14] (pp. 264 and 213) one of the solutions is 

(22) Uiz) - Kiiz), 


* 1 profit of this occasion to thank him for this and other valuable suggestions. 
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where Ki{z)y the modified Bessel function of the second kind (Hankelfunction) 
is defined by 


(23) 


Ki( 2) = (7 - Ig 2 +.lg z) 


00 1 

V- - - 

^v\{v + 1)1 




Z 

1 


1 

{v — 1)! v! 



+ 




The relation between the functions K,(2) and the Hankelfunction is 


(23a) K,(a) - ? 

The as3anptotic probability for the range is, from (20) and (22), 

(24) ^{R) = zKi{z) 
or, from (19) 

(25) 'J'(ff) = 2e~‘'*Kii2e-‘'^). 

This is the only Bessel function satisfying the boundary conditions (21). The 
asymptotic probability ^(R) of the range may be written finally from (25), (23) 
and (10) 

(25a) 1 - ’*'(«) = i; (fi - 27 + 2S, - -) 

1 J'l — 1)1 \ v/ 

where 

-So = 0; -S, = E I 

X«1 A 

The distribution 

ct^iR) dz 

iK® - “5" • 3* 

of the reduced range R is, from (24) and (19) 

m) = + zK[{t)). 

Now, the derivative Ki{z) is linked to the modified Bessel function K^{z) of 
the second kind and of order zero by 

zK[{z) = -Ki{z) - zK,{z). 

Consequently the distribution is 

( 26 ) ^^iR) ^ ^ Koiz) 
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or, from (19), 

(27) 4>{n) = 2e-^Ko(2e-^'^) 
where the function Ko{z) is defined by 

( 28 ) K„(z) = - (7 - Ig 2 + Ig 2) 

Finally the asymptotic distribution ypiR) of the reduced range may be written 
from (27) and (28) 


‘+2+ 




(28a) 


HR) = E --- t {R-2y + 2Sr) 
0 I'! vl 


We first investigate the analytic behavior and the order of magnitude of the 
probability '^{R) and the distribution \I/(R) for large negative, and large positive 
values of the reduced range, i.e. for large and small values of the positive variable 
2. If z is so large that 

( 3 «/ 2 ) 

(29) 2“ * = « 1 


the expressions for ^1(2) and Ko(z) become [14], p. 271, 

Ki(^) = y^^®'(^+ 82“ 1^^) 

® “ 82 1282*) 

The probability ^{R) becomes, from (24) and (19), 

(250 ^(K) = exp[- f - 2e-'»'«] (l +1 ^ • 

The condition (29) holds, say, for = — 4. The numerical calculation leads, 
for 'i'(“4), to the order of magnitude 10“®. 

In the same way, the distribution \l/(R) becomes, from (26) and (19), for large 
negative reduced ranges 

(27') m - Vi e.p[-M - (._*“ + ,•) . 

This expression cannot be ‘obtained from (25') since the approximations for 
^0(2) and Ki{z) used do not fulfill the relations between the derivatives given 
above. The order of magnitude of ^(—4) is 10“*^. 

Thus the probability ^(i?) and the distribution ^(J?) may be neglected for 
B g —4. This removes the importance of the lower limit R ^ —2cm stated 
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in (10'). If otu ^ 2, the distribution of the range may be dealt with as if it 
were practically unlimited toward the left. 

For large positive reduced ranges to which correspond small values of z, say 

(290 « 1 

the Bessel functions Ki(z) and lUiz) become, from (23) and (28) 

(23-) «.) - (. + Ig 0 (i+Q + l-(t + I’) 

(28') K.(2) . + lg|) + i + ^ . 

In this case we are interested to know how far the probability ^(i^) differs from 
unity. Consequently we calculate 1 — ^{R) and obtain, from (24) and (23') 

1 - MK) = -^[(^7 + +0 - 1 _ I], 

The right side becomes, from (19) 

2.-[(-, + f)(l+9 + l + |e-] 

= e-* -2y)(l+^^ + i+ 

or 

c-“|^K - 27 + 1 + - 27 + 0]= c-*(B - 27 + 1)^1 + • 

If R is so large that 

e """«1 


we simply have 

(25") 1 - ^R) = e'^iR ~ 27 + 1) . 

For example, for R = 10, the preceding condition is satisfied and 1 — 4^(jR) is 
of the order 5.10“^ 

In the same manner we calculate the density of probability ip^R) for large 
reduced ranges. From (26), (19) and (28') we obtain 

m = 2e-* [(I - 7) (1 + e-«) + e-“ + . 

By neglecting e”** <K R, the right side becomes 

e“*[(ft - 27 X 1 + e"*) + 2e-*] = - 2y + e'^R -27 + 2)] 

whence 


m = e-^iR - 27)(1 + e-*) + 2«-‘*. 
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In first approximation we obtain 

(27") m) = e^\R - 27 ) 


a formula which may also be derived directly from (25"). The density of 
probability is of the order 10“^ for R = 10. 

From the formulae (25') and (27') valid for large negative values of R, and 
from the formulae (25") and (27") valid for large positive values of R follow the 
boundary conditions 


lim 

Ai»—00 


mi ^ 
MR) 




m _ 

1 ~ MR) R-2y + l 


For the construction of tables of the distribution ^(7?) and the probability 
'^{R) of the reduced range it is sufficient to consider the interval 


-S <R < 10. 


The two functions Ki(z) and Koiz) have been tabulated [14] and [19]. Hence 
the probability and the distribution could be calculated from such tables of the 
Bessel functions. Tliis procedure, however, was only used to obtain boundary 
values. The tables I and la arc based on computations made in the Calculation 
and Ballistics Department at the Naval Proving Ground Dahlgren by stepwise 
integration of the differential equation (17) using the special Relay Calculator 
of the International Business Machines Corporation.® 

Table I gives the probability ^(7?) (col. 2) and the distribution \p{R) (col. 4) 
for the reduced ranges —3 ^ 7? g 10.5 in intervals AR = 0.5. The differences 
given in col. 3 are taken from the original figures. 

For different uses it is necessary to know the reduced range as a function of 
its probability. This relation is shown in Table la. The first column gives the 
probability, the first line gives the last decimal of this probability, and the cells 
give the reduced range corresponding to the probability obtained from the 
combination of the first column and the first line. For example: The reduced 
range R = —3.20 corresponds to the probability ^(7?) = 0.0002, and the reduced 
range R = 10.44 corresponds to the probability 'i'iR) = 0.9997. 

This table may be used for obtaining the percentage points of the reduced 
range. The mode -ft, the median ft calculated by the Naval Proving Ground 
and the mean ft obtained from (14) and (10) are 

(30) ft = 0.506366440; ft = 0.928597642; ft = 1.154431330. 

A probability paper for the range may be constructed in the following way: The 
observed ranges w are plotted on the vertical axis; the reduced ranges R on sl 
horizontal axis. The abscissa shows the probabilities 

^^^(72) = G{w) 

* The author wishes to express his sincere appreciation for the permission to use these 
computations. The original tables give the probability and the distribution to 8 significant 
decimal places at intervals AR -■ 1/100. Lack of space prevents the reproduction of these 
tables. 
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TABLE I 


Asymptotic Probability and Asymptotic Distribution of the Reduced Range 


1 

2 

3 

4 

Reduced Range 

Probability 

Difference 

Distribution 

R 

« (R) 


^(R) 

-3.0 

.00050 

.00274 

.00212 

-2.5 

.00324 

.01032 

.01057 

-2.0 

.01356 

.02693 

.03386 

-1.5 

.04048 

.05251 

.07705 

-1.0 

.09299 

.08141 

.13419 

- .5 

.17440 

.10533 

.18969 

.0 

.27973 

.11821 

.22779 

.5 

.39794 

.11859 

.24075 

1,0 

.51654 

.10891 

.23021 

1.5 

.62545 

.09327 

.20346 

2.0 

.71872 

.07557 

.16898 

2.5 

.79429 

.05860 

.13360 

3.0 

.85289 

.04386 

.10157 

3.5 

.89675 

.03192 

.07483 

4.0 

.92867 

.02270 

.05375 

4.5 

.95136 

.01584 

.03783 

5.0 

.96721 

.01089 

.02618 

5.5 

.97810 

.00739 

.01787 

6.0 

.98549 

.00496 

.01205 
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TABLE 1—Concluded 


1 

2 

3 

4 

Reduced Range 

R 

Probability 

«'(R) 

Difference 

Distribution 
^ (R) 

6.5 

.99045 

.00330 

.00805 

7.0 

.99375 

.00218 

.00534 

7.5 

.99594 

.00143 

.00351 

8.0 

.99737 

.00093 

.00230 

8.5 

.99830 

.00061 

.00150 

9.0 

.99891 

.00039 

.00097 

9.5 

.99930 

.00025 

.00062 

10.0 

.99955 

.00016 

.00040 

10.5 

.99972 


.00026 


corresponding to the reduced ranges R, If the observations follow the theory, 
the observed ranges are scattered around the straight line 

(10') ic = 2w + - 


If the samples are drawn simultaneously, and if there is a constant interval of 
time between the drawings, this interval may be used as unit of time for the 
construction of the return periods T{R) and iT{R) of a range equal to, or larger 
than (smaller than) R where 


nR) = 


1 

1 - '^{R) 


; iT{R) 


1 

^{R) ' 


The first (second) notion applies to the range above (below) the median. The 
return periods are shown in an upper parallel to the abscissa. 

A scheme for this paper is given in Fig. 3. Such a paper will allow a graphical 
test for the fit of the observed ranges to our theory, and avoids any numerical 
calculations. Obviously this method may only be used if the initial distribution 
is symmetrical, unlimited, and of the exponential type, and if the sample size 
is so large that the asymptotic distribution holds. 
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6. The range, the midrange, and the extremes. The asymptotic* dis¬ 
tribution (27) of the reduced range was obtained by convolution of the asympto¬ 
tic distributions (5) of the extremes. The same method leads to the asymptotic 
distribution of the reduced midrange [8] 

(31) V = ot{xi + itn). 

TABLE lA 


The Reduced Range R as Function of Its Probability ^ (R) 


♦ (R) 

0 

1 

2 

3 

4 

5 

6 


8 

9 

.000 

— 

* 

-3.20 

-3.12 

-3.05 

-3.00 

-2.96 

-2.92 

-2.89 

-2.86 

.00 

- 

-2.83 

-2.64 

-2.52 

-2.43 

-2.36 

-2.30 

-2.25 

-2.20 

-2.16 

.0 

— 

-2.12 

-1.84 

-1.65 

-1.51 

-1.39 

-1.28 

-1.19 

-1.10 

-1.02 

.1 

-0.05 

-0.88 

-0.81) 

-0.75 

-0.69 

-0.63 

-0.58 

-0.52 

-0.47 

-0.42 

.2 

-0.37 

-0.32 

-0.27i 

-0.22 

-0.18 

-0.13 

-0.09 

-0.04 

0.00 

0.04 

.3 

0.09 

0.13 

0.17 

0.22 

0.26 

0.30 

0.34 

0.38 

0.43 

0.47 

.4 

0.51 

0.55 

0.59 

0.63 

0.68 

0.72 

0.76 

0.80 

0.84 

0.89 

.5 

0.93 

0.97 

1.02 

1.06 

1.10 

1.15 

1.19 

1.24 

1.28 

1.33 

.6 

1.38 

1.43 

1.47, 

1.52 

1.57 

1.62 

1.67 

1.73 

1.78 

1.84 

.7 

1.89 

1.95 

2.01' 

2.07 

2.13 

2.19 

2.26 

2.33 

2.40 

2.47 

.8 

2.54 

2.62 

2.70 

2.79 

2.88 

2.97 

3.07 

3.18 

3.29 

3.41 

.9 

3.54 

3.69 

3.85 

4.03 

4.23 

4.46 

4.75 

5.11 

5.61 

6.45 

.99 

6.45 

6.57 

6.71 

6.87 

7.05 

7.26 

7.52 

7.85 

8.31 

9.10 

.999 

9.10 

9.22 

9.35 

9.50 

9.67 

9.881 

10.12 

10.44 

* 

♦ 


* These values have not been calculated. 


On the other hand, the asymptotic distributions of the reduced extremes are 
obtained by introducing the transformations 

(32) 2/1 = aixi + u); 2/n = a(a^n ~ u) 

into formulas (5). It is interesting to compare these four distributions and four 
probabilities with each other. Thi| is done in Figures 1 and 2. The probability 
and the distribution of the midrange are practically identical with the probability 
and distribution of the smallest value, for small values of the midrange, and 
become practically identical with the probability and distribution of the largest 
value for large values of the midrange. Fig. 2 shows that the asymptotic dis¬ 
tribution of the reduced range is less asymmetrical than the asjmptotic distribu¬ 
tions of the reduced extremes. 
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Table II contains some characteristic values for these four asymptotic dis¬ 
tributions. The first three columns are obtained from previous publications 
[6, 8]. The mean range is equal to the range of the means for the extremes. 
The median of the range is larger than the range from the median of the largest 
to the median of the smallest value. The mode of the range is shghtly smaller 
than the mean of the largest value. These statements hold, ot course, only^for 
the reduced variates. 



Fig 2 

From the mode & of the reduced range given in equation (30) and the trans¬ 
formation (10), the mode w of the range itself is obtained as 

w = 2u — 
a 

whereas the difference of the modes of the largest and of the smallest values is 

Xn — xi = 2u. 

V) = Xn — Xi + - . 

a 


Consequently 

( 33 ) 





400 


£!• J* OUAfSULi 



Reoucu> 






DISTRIBUTION OP RANGE 


401 


For a symmetrical initial distribution of the exponential type the mode of the 
range converges toward the range of the modes of the smallest and of the largest 
value, provided that the parameter a increases without limit with the sample 
size. Thus this convergence does not hold for all symmetrical distributions. 

The last two lines in Table II give the four probabilities corresponding to the 
intervals from the mean /x minus once (twice) the standard deviation <r, up to 
the mean plus once (twice) the standard deviation. The first probability for 

TABLE II 


Characteristics for the 4 Asymptotic Reduced Distributions 


1 

Characteristic 

2 

Largest Value 

3 

Smallest Value 

4 

Midrange 

5 

Range 

Mode 

0 

0 

0 

.506 

Expectation 

7 - .57722 

= -.57722 

0 

27 « 1.15444 

Median 

-lglg2 = .36651 

= -.36651 

0 

.929 

Seminvariant char, 
function 

r(l - 0 

r(i + 0 

r(i- t)-r(i + t) 

r^i - 0 

Variance 

J = 1.64493 

« 1.64493 

IT* 

3 

« 3.28986 

First -f second mo¬ 

/3i = 1.29857 

-1.29857 

0 

.64928 

ment quotient 

li 

5.4 

4.2 

4.2 

95% Probability 

2.97 

1.10 

2.94 

4.46 

99% Probability 

4.60 

1.53 

4.60 

6.45 

+ 

1 

1 

.72 

.72 

.72 

.71 

-f 2a) — F(ti — 2a) 

.90 

.90 

.95 

.95 


the four distributions is about the same as for the normal distribution. The 
second. probability for the range and the midrange is about the same as for 
the normal one. 

7. The asymptotic distribution of the range for a symmetrical variate. 

The asymptotic distribution of the range R is, of course, independent of 
the sample size, and parameter-free. Both statements do not hold for the 
distribution g{w) of the range proper which is, from (11) 

(34) g(w) = oaplaiw - 2w)]. 

In this formula, the range is expressed in the same units as the initial variate. 
The parameters a and u are functions of the sample size n, the function depending 
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upon the initial distribution. From equations (6), (8), (14) follows that an 
increase of the sample size has two influences on the distribution of the range. 
The increase of the parameter u shifts the distribution toward the right without 
changing its form, whereas the parameter a influences the shape of the distribu¬ 
tion. If a increases (decreases) with n, the distribution of the range shrinks 
(spreads) with increasing sample size. If a is independent of n, an increase of 
the sample size does not change the shape of the distribution. Only in the first 
case may we increase the precision of the range by increasing the sample size. 
The two parameters thus influence the range in the same way as they influence 
the extreme values. 

To use equation (34) for a given initial distribution and a given sample size, 
we have to determine the expected largest value u and the parameter a as func¬ 
tions of n. Wc may use the definitions (6), (7), (8) if the initial distribution is 
known and of the exponential type, and if the sample size is so large that the 
most probable largest value is sufficiently near to the solution of (7). 

As a first example, consider the so-called logistic distribution. This prob¬ 
ability is 

(35) Hx) = (1 + 

The initial distribution is 

(350 <p(x) = ^x){l - ^(x)) 

and the derivative is 

(35'0 ^'(x) = 4>(a;)(l - ^{x))il - 24>(x)). 

Equation (6) becomes 

n — 1 

whence the expected largest value 

(36) u = lg(n - 1). 

The most probable largest value Xn for n observations is obtained from (7). 
This equation becomes, from equation (35) 

, (n - 1)(1 - Mxn)) = -1 + 2Hxn) 


whence 4>(xn) = —^ 

n + 1 

Equation (35) leads to the most probable largest value 
(360 Xn^ Ign. 

Even for n as small as 30 the difference between Xn and u is less than 1%. Con¬ 
sequently the asymptotic form of the distribution of the range may be used even 
for small samples. The two parameters are 
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Since a converges toward unity, an increase of the sample size shifts the distribu¬ 
tion of the range toward the right without influencing its shape: the precision of 
any estimate made from the range cannot be increased by increasing the sample 
size. 

The characteristic ranges introduced in paragraph 5 are obtained immediately: 
the mean fZ), the mode w, the median range w and the ranges Wm and Wm 

W = \gn + 1,154; w = \gn +.506; 

u) = Ig n + .929; Wm = Ig n + 4.46; 1^.99 = Ig n + 6.45 

are parallel straight lines if traced as functions of the sample size n on semi- 
logarithmic paper. 

For the normal distribution we cannot expect such simple results. Here, u 
and a can only be calculated as numerical functions of n although limiting forms 
of these functions are known. The parameter a increases with n, and the 
standard error of the range decreases without limit although very slowly. The 
logistic distribution belongs to the first, the normal distribution to the second 
class of initial distributions of the exponential type. 

The probabilities and the distributions of the range for normal samples of 
size 5, 10, and 20 as calculated by E. S. Pearson and H. 0. Hartley [16] are 
traced in Figures 4 and 5. Our aim is to trace the corresponding asymptotic 
probabilities and distributions in order to see how far the asymptotic ranges 
differ from the exact ones. However, we have first to settle the preliminary 
question how far the most probable largest value Xn differs from the expected 
largest value u. The most probable largest value Xn is obtained from (7) which 
becomes, for the normal distribution, 

(38) Xn^Xn) = (n - l)^(Xn). 

The results Xn as functions of n are shown in Table III cols. 1 and 2 . The 
expected values u obtained from ( 6 ) are given in col. 3. For small samples, the 
two values Xn and u differ widely, as might be expected. We are inclined to 
conclude that the asymptotic distribution of the range cannot hold for small 
samples. However, the only legitimate conclusion to be drawn is, that we can¬ 
not calculate the two parameters in the w^ay stated before ( 6 ) and ( 8 ). Instead, 
we estimate them directly from the observations. The question of the most effi¬ 
cient estimates of these parameters is not yet solved. The simplest way is to 
use the mean range Wn and the standard deviation of the range (rtv.n as given by 
Tippett [20] and Pearson [15]. To distinguish these estimates from the asympto¬ 
tic values, we write the estimates with an index n. From (14) we obtain 

(39) Ji = o-„,„; 2Un=V>n-^. 

an TT an 

Table III gives the calculated means Wn and standard deviations o-^.n of the 
range, and the estimates 1/an and 2 w« . Fig. 6 shows how the most probable 
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largest values Xn approach the expected largest value u with increasing sample 
size. The estimate Un quickly approaches u. Besides we trace the mean range 
On , the standard error of the range o-«,.n , and I/an which is proportional to it. 



From col. 8 follows that the condition ait ^ 2 is fulfilled from n ^ 6 onward. 
The ranges obtained from the transformations 

R 

(40) w = 2un + 

an 

are given in Table IV, cols. 3-7. The asymptotic probabilities of the range as 
obtained from the combination of columns 3-7, and col. 2 of Table IV are traced 
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in Fig. 4 as separated points. The asymptotic probabilities are situated very 
near to the exact ones. Therefore the same method was used to calculate the 
asymptotic probabilities of the range for n = 50 and n = 100 which have not 
been calculated by Pearson. They too are traced in Fig. 4. 



The asymptotic probabilities of the range hold even for small normal samples. 
However, the parameters obtained from the exact distribution differ considerably 
from their asymptotic values. In other words: The asymptotic probabilities of the 
range hold even for small normal samples provided thcU the parameters are taken 
from the observations. 

To compare the asymptotic distributions of the normal range to the calculated 
distributions, we attribute the asymptotic differences A^/an for a unit interval 
Aw? = 1 to the middle of the corresponding intervals. The results are traced in 
Fig. 5 f orn = 5, 10, 20, 60, 100. On the other hand, we take the differences 
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for unit intervals from Pearson's tables, and trace them in the same graph. 
The fit of the calculated to the asymptotic values may be considered satisfactory. 

TABLE III 

Estimate of Parameters from the Calculated Distributions 
of the Normal Range 


1 

2 

3 

4 

Sample 
size n 

Largest Value 

Mean Range 

V\ n 

Modal 

Expected 

u 

3 

.765 

.431 

1.693 

4 

.938 

.674 

2.059 

5 

1.061 

.842 

2.326 

10 

1.419 

1.282 

3.078 

20 

1.740 

1.645 

3.735 

50 

2.126 

2.054 

4.498 

100 

2.377 

2.326 

1 5.015 


5 

6 

7 

8 

Standard 

deviation 

Estimated parameters 
of the range 

Lower 

limit 

<^w,n 

l/ttn 

2Un 

2anUn 

.8884 

.4898 

1.128 

2.30 

.8798 

.4851 

1.499 

3.09 

.8641 

.4764 

1.776 

3.73 

.797 

.439 

2.571 

5.86 

.729 

.402 

3.271 

8.14 

.653 

.360 

4.082 

11.34 

.605 

.334 

4.630 

13.86 


TABLE IV 


Asymptotic Probabilities for Normal Ranges Taken from Small Samples 


1 

2 

3 

4 

5 

6 

7 

Reduced 

range 

R 

Probability 

Normal ranges w 

== 2un + R/an for sample sizes 

G(w) * SI^(R) 

n-5 

n « 10 

n » 20 

n — 50 

n 100 

-3 

.000 

.35 

1.25 

2.07 

3.00 

3.62 

-2 

.014 

.82 

1.69 

2.47 

3.36 

3.96 

-1 

.093 

1.30 

2.13 j 

2.87 

3.72 

4.30 

0 

.280 

1.78 

2.57 

3.27 

4.08 

4.63 

1 

.517 

2.52 

3.01 

3.67 

4.44 

4.96 

2 

.719 

2.73 

3.45 

4.07 

4.80 

5.30 

3 

.853 

3.21 

3.89 

4.48 

5.16 

5.63 

4 

.929 

3.68 

4.33 

4.88 

5.52 

5.97 

5 

.967 

4.16 

4.77 

5.28 

5.88 

6.30 

6 

.985 

4.63 

5.20 

5.68 

6.24 

6.63 

7 

.994 

5.11 

5.64 

6.09 

6.60 

6.97 


Fig. 5 shows furthermore how the distributions of the range are shifted toward 
the right and become more concentrated for increasing sample sizes. 

As an example for the practical application of the asymptotic distribution of 
the range, we use an observed distribution of 50 ranges taken from samples of 
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w = 14 normal values given in Freeman’s book [5] p. 128. The observed step 
function is traced in Fig. 7. For reasons given in a previous article [7] we 
attribute the ciunulative frequency .5 to the smallest range 3, and the cumulative 
frequency 49.5 to the largest range 18. To compare this step function with the 



4 a IZ. 16 SO 

Fig. 7 


probability G{w), we estimate the two parameters Un and an from formula (39). 
TheJ^mean range Wn and the estimate Sw,n of the standard deviation of the ranges 
are 

© = 10.68; = 2.93. 

Consequently we obtain, from (39) 

~ = 1.61; 2un = 8.82. 
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The theoretical ranges are thus, from (40), 

w = 8.82 + 1.61 R. 

The corresponding probabilities G(w) taken from Table I are traced in Fig. 7. 
The fit of the theory to the observations is certainly satisfactory, especially if 
we take into account that the ranges are given in integer numbers only. 

8. The mth range and the asymmetrical case. An obvious generalization 
of the theory as established in paragraph 4 consists in the construction of the 
asymptotic distribution of the mth range for an unlimited symmetrical distribu- 
tion of the exponential type. The mth range is the positive distance from the 
mth observation from above, Xm , to the mth observation from below, mX, We 
suppose m to be very small compared to the sample size. Under the conditions 
stated in the beginning, the joint distribution Xm) of the mth extreme 

values splits into the product of the asymptotic distribution of the 7nth extreme 
value from above, fm(xrn)j by the asymptotic distribution of the mth extreme 
value from below, mf(mx). Here, [6] 

ftniXm) = am CXp [ — mafn(x^ ~ ?/m) “ mt 
mfimX) = am CXp [mamimX + Wm) ~ 

The sample size must be so large that the most probable mth extreme value Xm 
is sufficiently near to Um which is defined as the solution of 

= 1 - ^ . 
n 


The factor am defined by 


__ 

” 1 - ^{Um) 


is related to the asymptotic standard error trm of the mth extreme value by 


am O’m 



The joint as 3 miptotic distribution Xm) of the mth smallest value and the 

mth range 

(41) Wm = Xm — mX 


is 


toimX, Wm) = am CXp [~ mamiWm ~ ^ 

The asymptotic distribution g(wm) of the mth range is, dropping the index m of 
the variable mX, 

g(wj = 1^ exp I- dx. 



410 


E. J. GUMBEL 


Again we introduce a reduced range Em defined by 

(42) OtmiWm “ 2Um) = Em ^ 
and put for the integration 

Otm (X + Um) = 2 /. 

Then the asymptotic distribution of the reduced mth range is 

(43) \l^{Em) = exp[—mt" — dy. 

The probability ^(/^n») for the mth range 

'^(Em) = f ypiz) dz 

•^2a„Un 

cannot be reduced to a single integral. This is due to the fact that the proba¬ 
bilities of the mth extreme values cannot be written down except in the integral 
form [6]. No differential equation similar to (17) exists. However, the function 
(43) could be calculated by numerical methods. The mean Em , the generating 
function and the moments of the mth range have been given in a previous 
paper [8]. 

For sake of completeness, consider finally an unlimited asymmetrical initial 
distribution of the exponential type. In this case, the joint distribution of the 
smallest and of the largest value splits again, for large samples, into the product 
of the asymptotic distributions/i(a;i) and f„(xn) of the smallest and of the largest 
values which are now [6] 

/,(*0 = «xexpk(x, - Mj) - 

= a„exp[-a„(x„ - «„) - e 

Here, an and Un are defined, as previously, by (6) and (8). The sample must 
be so large that the most probable smallest value x, is sufficiently near to the 
solution of 

= - . 
n 

The factor ai defined by 

v>(^i) 

4>(wi) 

is related to the asymptotic standard error of the smallest value by 

The joint asymptotic distribution of the smallest value Xi and the range w 
h)(xi, w) = aittn exp[ai(a:i — Wi) — anixy + w — Un) — 
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contains four parameters instead of the two which exist in the symmetrical case. 
However, the number of parameters may be reduced to one. We introduce a 
reduced range R defined by 

(44) R = an{w — Un + Wl) 

being the range itself minus the range of the modes divided by a factor pro¬ 
portional to the standard error of the largest value. If we put 

(45) ai(a;, - Mi) = y; — = /J 

0(1 

the distribution 4>{R) of the reduced range becomes, in the asymmetrical case, 

(46) l^(ff) = e~® { exp( 3 /(l — 0) — e’' — e"'’’'”®] dy 

J—QO 

and the probability ^(i2) for the reduced range is 

(47) ^(R) = j exply — 6*' — dy 

a formula which may immediately be verified by differentiation with respect to 
R, The mode R of the range is the solution of 

\I/{R) = c~^ exp[ 2 /(l — 20) - R — dy. 

Contrary to the symmetrical case, the latter integral cannot be expressed by the 
probability, and no simple differential equation similar to (17) exists. The ex¬ 
pressions (46) and (47) contain a single constant ^ measuring the asymmetry of 
the initial distribution. In the symmetrical case, jS = 1, we obtain, of course, the 
previous formulas (12) and (13). In the asymmetrical case, the mean, the 
variance, and the higher moments of the mth range may be derived from the 
generating function given in a previous paper [8]. 

The asymptotic distribution of the mth range in the asymmetrical case can 
easily be obtained by combining the two procedures used in this paragraph. 
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Addition at proof reading: 

G. Elfving’s article “The asymptotical distribution of range in samples from a normal 
population”, Biometrika, Vol. 35 (1947), appeared w-hen this manuscript w'as ready for 
print. Elfving considers a probability transformation of the range whereas we deal with 
the range itself. His distribution requires the knowledge of the initial distribution and 
of the sample size, whereas this knowledge is not required in our asymptotic formula. 
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1. Siunmary. The means, variances, and covariances for samples of size 
< 10 from the normal distribution, a selected long-tailed distribution, and the 
uniform distribution are tabled and compared with the usual asymptotic ap¬ 
proximations. The methods of computation used and the accuracy expected 
are discussed. Use is made of the representation of an arbitrarily distributed 
variate as a monotone function of a uniformly (rectangularly) distributed vari¬ 
ate. It is hoped that these tables will encourage experimentation with new 
statistical procedures. 

2. Introduction. Two sorts of statistical procedures have been widely ex¬ 
ploited in theoretical statistics—^first the use of linear and quadratic combina¬ 
tions of the unordered observations and, second, the use of ranked (ordered) 
observations. Statistics based on ordered observations have recently been 
dubbed systematic statistics [2, Mosteller, 1946], Analytic processes and a few 
necessary numerical tables have advanced the study of the first procedure greatly, 
at least for the special case of the normal distribution; but analytic procedures 
have not done much to exhibit the behavior of systematic statistics and the neces¬ 
sary tables have been lacking. 

It would be very helpful to have (1) at least the first two moments (including 
product moments) of the order statistics, and (2) tables of the percentage points 
of their distributions, for samples of sizes from 1 to some moderately large value 
such as 100 and for a large representative family of distributions. This is a 
large order and will require much computation. 

The first step in this direction was taken by Fisher and Yates [1] by tabulating 
the means, to two decimal places, of all order statistics from normal samples of 
size < 50. The present paper continues the process by supplying all means, 
variances, and covariances for samples of size < 10 from (a) the normal dis¬ 
tribution, (5) the uniform (rectangular) distribution, (c) a special distribution 
with long tails. For purposes of comparison, we also supply approximate 
means, variances, and covariances for the uniform and the special distribution 
computed from suitable asymptotic formulas. 

The special distribution has the representing function 

(1) r{u) = (1 - 

where u has the uniform distribution on the interval [0, 1], and x = r{u) is the 
variable whose order statistics interest us. This special distribution was es- 
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pecially constructed 1) to have high tails and 2) to provide moments of order 
statistics in closed form which could be evaluated with a reasonable amount of 
labor. The normal distribution is rather unreasonable in this latter respect— 
there being no known expression except in terms of single and double quadra¬ 
tures of some considerable numerical difl&culty. 

We have restricted ourselves to samples of size < 10, and to only three dis¬ 
tributions, all of these symmetrical, because of limited man-power rather than 
limited interest. Additional tables of a similar nature will surely prove helpful. 

In order to obtain even as much information as provided in this paper, it has 
been necessary to make a joint effort, dividing the labor. The various parts of 
the work have been carried out more or less separately by the various authors— 
the means and variances for the normal by Hosteller, the covariances for the 
normal (which, with their double quadratures, required far more time than all 
the other thought and computation combined) by Hastings with some assistance 
from Hosteller, the choice of the special distribution by Tukey, and the com¬ 
putation for it by Winsor. 


3. Results. In this section we provide the various tables that have been 
computed. 

Table I gives the mean and standard deviation of the /th order statistic 
x{i I n), [or a;<,n, we use whichever notation seems less likely to confuse and 
agree that x(l | n) > x(2 | n) > • • • > x(n [ ?0] from a sample of size n drawn 
from a uniform (U), normal (A), and a special distribution (S). All three 
distributions have been adjusted to have zero mean and unit variance. In 
addition Table I gives approximations for the mean and standard deviation as 
computed from asymptotic formulas for the normal {AN) and the special {AS), 
If f{x) is the density function, the asymptotic approximation for the mean 
m{i I n) of the eth order statistic from a sample of size n is obtained by solving 
the equation 



dx = i/{n + 1) 


for m{i I n). Similarly the formula used for the asymptotic variance of x{i | n) 
is 

_ i {n — i + 1) 

n{n + l)*{/[w(iln)]l^' 

Values are given for n = 1, 2, • • • , 10 and i = 1, • • • , If w (f| n) is 

an entry in the table for means, a missing entry m{n — i + 1 | n) = — m(i | n); 
if w{i I n) is an entry in the table of standard deviations, a missing entry 

w{n — t -h 11 n) = w(i | n). 

Table II gives the variances and covariances of the order statistics for the 
normal distribution (AT) and the same quantities as approximated by the asymp- 
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TABLE I 


Means and standard deviations of order statistics x{i\n) for uniform distribution 
{U)j normal (AT), special (aS), asymptotic normal (AAT), 
asymptotic special (A/S) 


Mean 

Standard Deviation 

n 

% 

U N S 

AN AS 

U N S 

AN AS 

1 

1 

0 0 0 

0 0 

1.00000 1.00000 1.00000 
1.2533 .9804 

2 

1 

.57735 .56419 .53493 

.4307 .3418 

.81650 .82565 .84490 

.9168 .7486 

3 

1 

.86603 .84628 .80240 

.6745 .5466 

.67082 .74798 .82783 

.7867 .6823 

2 

0 0 0 

0 0 

.77460 .66983 .58457 

.7236 .5660 

4 

1 

1.03923 1.02938 .98473 

.8416 .6954 

.56569 .70122 .82982 

.7144 .6542 

2 

.34641 .29701 .25540 

.2533 .1992 

.69282 .60038 .52582 

.6340 .5035 

5 

1 

1.15470 1.16296 1.12449 
.9674 .8136 

.48795 .66898 .83642 

.6670 .6415 

2 

.57735 .49502 .42567 

.4307 .3418 

.61721 .55814 .50390 

.5798 .4730 

3 

0 0 0 

0 0 

.65465 .53557 .44903 

.5605 .4384 

6 

1 

i 

1 

1 

1.23718 1.26721 1.23847 

1.0676 .9114 

.42857 .64492 .84423 

.6331 .6330 

2 

.74231 .64176 .55458 

.5659 .4539 

.55328 .52874 .49425 

.5426 .4567 

3 

.24744 .20155 .16785 

.1800 .1412 

.60609 .49620 .41648 

.5147 .4057 
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TABLE I {Continued) 


Mean 

1 standard Deviation 

n 

t 

U N S 

AN AS 

U N S 

AN AS 

7 

1 

1.29904 1.35218 1.33506 
1.1504 .9957 

.38188 .62603 .85217 

.6072 .6141 

2 

.86G03 .75737 .65892 

.6745 .5462 

.50000 .50670 .48992 

.5150 .4359 

3 

.43301 .35271 .29375 

.3186 .2512 

.55902 .46875 .39963 

.4826 .3772 

4 

0 0 0 

0 0 

.57735 .45874 .37747 

.4737 .3617 

8 

1 

1.34715 1.42360 1.41892 
1.2207 1.0697 

.34427 .61066 .85988 

.5867 .6276 

2 

.96225 .85222 .74690 

.7647 .6259 

.45512 .48930 .48823 

.4936 .4402 

3 

.57735 .47282 .39498 

.4307 .3418 

.51640 .44807 .38998 

.4584 .3743 

■ 

.19245 .15251 .12502 

.1397 .1091 

.54433 .43264 .35616 

.4447 .3494 

9 

1 

1.38564 1.48501 1.49358 

1.2816 1.1358 

.31334 .59780 .86725 

.5691 .6268 

2 

1.03923 .93230 .82317 

.8416 ' .6954 

.41779 .47508 .48800 

.4763 .4361 

3 

.69282 .57197 .47995 

.5244 .4191 

.47863 .43171 .38414 

.4393 .3722 

4 

.34641 .27453 .22504 

.2533 .1992 

.51168 .41303 .34321 

.4227 .3356 

5 

0 0 0 

0 0 

.52223 .40751 .33173 

.4178 .3268 
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TABLE I {Concluded) 


Mean 

Standard Deviation 

n 

i 

U N 

AN AS ^ 

U N S 

AN AS 

10 

1 

1.41713 1.53875 1.56057 
1.3352 1.1956 

.28748 .58681 .87423 

.5557 .6275 


2 

1.10222 1.00135 .89062 

.9085 .7574 

.38569 .46318 .48859 

.4619 .4334 


3 

.78730 .65608 .55336 

.6046 .4866 

.44536 .41826 .38054 

.4238 .3604 


4 

.47238 .37572 .30866 

.3488 .2754 

.48105 .39756 .33477 

.4052 .3261 


5 

.15746 .12274 .09961 

.1142 .0894 

.49793 .38857 .31190 

.3973 .3117 


totic formulas {AN), The asymptotic covariance between x{i | n) and x{j | n) 
is given by 

j{n - i + 1) . . 

n{n + iff[m{i | n)]f[m{j | n)] ’ •? - 

Symmetry relations exist for supplying the missing entries, 

cov [x{i I n), x{j I n)] = cov [x{n — i + 1 | n), x{n — i + 1 | n)]. 

It might seem more natural to use the factor w + 2 rather than n in the denomi¬ 
nator of the asymptotic variances and covariances so that the formulas would 
more nearly agree with those for the uniform distribution. However the use of 
n gives much better approximations for the normal and the special distribution. 

Table III gives the variances and covariances of the order statistics for the 
uniform distribution (17), and Table IV gives the corresponding results for the 
special distribution {S), Table V gives the asymptotic variances and co- 
variances for the special distribution (A<S). 

Table VI compares the correlation coefficients between the order statistics 
x{i 1 n) and x(j | n) for the uniform (17), the normal (V), and the special dis¬ 
tribution {S). 

It seems worthwhile to call attention to the following: 

(1) . Even for n = 10, the asymptotic formulas do not give satisfactory mean 
values for the order statistics. 

(2) . For n > 8, the asymptotic standard deviations for the normal are close 
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enough to be very useful. For the special distribution we must except the two 
order statistics on each end from this statement. 

TABLE II 


Variances and covariances of §ie order statistics j(t|n) for the 
normal (N) and the asymptotic normal {AN) 



< 

1 

2 

3 

4 

L ’ 

6 

7 

8 

9 

10 

n 























i 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

N 

AN 

2 

1 

.68 

.84 

.32 

.42 

















3 

1 

.56 

.62 

.28 

.33 

.17 

.21 
















2 



.45 

.52 

















4 

1 

.49 

.51 

.2\ 

.28 

.16 

.18 

.11 

.13 














2 



.36 

.40 

.24 

.27 















5 

1 

.45 

.44 

.22 

.24 

.15 

.17 

.11 

.12 

.07 

.09 












2 



.31 

.34 

.21 

.23 

.15 

.17 














3 





.29 

.31 















6 

1 

.42 

m 

.21 

.22 

.13 

.15 

.11 

.12 

.07 

m 

■1 

■s 










2 



.28 

.29 

.19 

.20 

.14 

.15 

Biti] 

.12 












3 





.25 

.26 

.18 

.20 













7 

1 

.39 

.37 

.20 

.20 

.13 

.14 

.10 

.11 

.08 

.09 

.06 

.07 

.05 

.05 








2 



.26 

.27 

.17 

.19 

.13 

.14 

.10 

.11 

.08 

.09 










3 





.22 

.23 

.17 

.18 

.13 

.14 












4 







.21 

.22 













8 

1 

.37 

.34 

.19 

.19 

.13 

.13 

.09 

.10 

.08 

.08 

.06 

.07 

.04 

.05 

.04 

.04 






2 



.24 

.24 

.17 

.17 

.12 

.13 

.10 

.10 

.08 

.09 

.07 

.07 








3 





.20 

.21 

.15 

.16 

.12 

.13 

.09 

.11 










4 







.19 

.20 

.15 

.16 











9 

1 

.36 

.32 

.18 

.18 

.12 

.13 

.09 

.10 

.07 

.08 

.06 

.07 

.05 

.05 

.04 

.05 

.04 

.04 




2 



.23 

.23 

.16 

.16 

.11 

.12 

.09 

.10 

.08 

.08 

.06 

.07 

.06 

.06 






3 





.19 

.19 

.14 

.15 

.11 

.12 

.10 

.10 

.08 

.08 








4 







.17 

.18 

.14 

,14 

.12 

.12 









■ 

B 

■ 

■ 

■ 

■ 

■ 

■ 
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1 

1 
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■ 

■ 

■ 

■ 

■ 

■ 
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10 

1 

1 

i 

i 

m 

i 

i 

m 

m 

m 

m 


m 

m 

1 


1 

ES 

i 

.03 

.03 


2 



.21 

.21 

.14 

.15 

.11 

.12 

.09 

.09 

.07 

.08 

.06 

.07 

.06 

.06 

.04 

.05 




3 





.17 

.18 

.13 

.14 

.11 

.11 

.09 

.09 

.08 

.08 

.06 

.07 






4 







.16 

.16 

.12 

.13 

.11 

.11 

.09 

.09 








5 






* 



.16 

.16 

.13 

.13 










(3). Forn > 8, the asymptotic variances and covariances of the normal are 
close enough for many, if hot most purposes. 





LOW MOMENTS FOR SMALL SAMPLES 


419 


(4). For the special distribution, only the variances and covariances of mod¬ 
erately central order statistics are adequately given by the as 3 nnptotic formulas. 

TABLE III 


Variances and covariances for the uniform distribution (U) 


n 

’\ 

1 

2 

3 

4 

s 

6 

7 

8 

9 

10 

2 

1 

.66667 

.33333 









3 

1 

.45000 

.30000 

.15000 









2 


.60000 









4 

1 

.32000 

.24000 

.16000 

.08000 








2 


.48000 

.32000 








5 

1 

.23810 

.19047 

.14286 

.09522 

.04762 







2 


.38095 

.28571 

.19047 








3 



.42857 








6 

1 

.18367 

.15306 

.12245 

.09184 

.06122 

.03061 






2 


1 .30612 

.24490 

.18367 

.12245 







3 


1 

.36735 

.27651 







7 

1 

.14583 

.12500 

.10417 

.08333 

.06250 

.04167 

.02083 





2 


.25000 

.20833 

.16667 

.12500 

.08333 






3 



.31250 

.25000 

.18750 







4 




.33333 







8 

1 

.11852 

.10370 

.08880 

.07407 

.05925 

.04444 

.02963 

.01481 




2 1 


.20741 

.17778 

.14815 

.11852 

.08889 

.05925 





3 



.26667 

.22222 

.17778 

.13333 






4 




.29630 

.23704 






9 

1 

.^9818 

.08727 

.07636 

.06645 

. 0.5455 

. 04.363 

.03273 

.02182 

.01091 



2 


.17455 

.15273 

.13091 

.10909 

.08727 

.06546 

.04363 




3 



.22909 

.19636 

.16364 

.13091 

.09818 





4 




.26182 

.21818 

.17456 






5 





.27273 






10 

1 

.08204 

.07438 

.06611 

.05786 

.04959 

.04132 

.03306 

.02479 

.01653 

.00826 


2 


.14876 

.13223 

.11670 

.09917 

.08264 

.06611 

.04959 

.03306 



3 



.19835 

.17356 

.14876 

.12397 

.09917 

.07438 




4 




.23140 

.19835 

.16529 

.13223 





6 





.24793 

.20661 






( 5 ), The correlation coefficients change rather little from distribution to dis¬ 
tribution, the poorest approximation being for end order statistics. 
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TABLE IV 


Variances and covariances for the special distribution (S) 


n 


\ 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

1 

.71385 

.28615 









3 

1 

.68530 

.24214 

.15957 









2 


.34172 









4 

1 

.68860 

.23277 

.14141 

.11123 








2 


.27649 

.17532 








5 

1 

.69960 

.23154 

.13655 

.10004 

.08614 







2 


.25391 

.15418 

.11490 








3 



.20163 








6 

1 

.71272 

.23310 

.13544 

.09667 

1 .07786 

.07080 






2 


.24429 

.14506 

.10486 

.08514 

i 






3 



.17345 

.12762 







7 

1 

.72619 

.23577 

.13582 

.09565 

.07517 

.06409 

.06042 





2 


.24002 

.14065 

.10004 

.07913 

.06776 






3 



1 .15970 

.11509 

.09134 







4 




.14249 







8 

1 

.73940 

.23890 

.13687 

.09562 

.07420 

.06179 

.05471 

.05291 




2 


.23837 

.13850 

.09754 

.07608 

.06359 

.05645 





3 



.15208 

.10822 

.08499 

.07138 






4 




.12685 

.10053 






9 

1 

.75211 

.24219 

.13825 

.09608 

.07398 

.06085 

.05266 

.04789 

.04721 



2 


.23814 

,13756 

.09625 

.07443 

.06141 

.05327 

.04852 




3 



.14756 

.10413 

.08097 

.06707 

.05835 





4 




.11780 

.09225 

.07680 






5 





.11004 






10 

1 

.76428 

.24550 

.13978 

.09680 

.07414 

.06053 

.05176 

.04604 

.04271 

.04272 


2 


.23872 

.13732 

.09565 

.07354 

.06018 

.05156 

.04594 

.04266 



3 



.14481 

.10158 

.07846 

.06444 

.05533 

.04940 




4 




.11207 

.08707 

.07180 

.06186 





6 





,10016 

.08300 






4. Methods of calculation and accuracy for the normal distribution. The 
means and variances of the order statistics for the normal distribution were ob¬ 
tained from direct quadrature of forms like 

r - F(x)r'~'/(x) dx, k = 1 , 2, 

J—00 

Fix) = ^ e-*'* dt and /(x) - F'(x). 


where 
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It is believed that the means are correct to within one unit in the fifth decimal 
and that the standard deviations are correct to within 2 or 3 units in the fifth 
decimal. 


TABLE V 


Variancea and covariances of the special distribution as computed 
from asymptotic formulas 


n 

t \ 

1 

2 

3 

4 

5 

6 

7 

8 

I 

9 

10 

2 

1 

.56044 

.28022 









3 

1 

.46550 

.22297 

.15517 









2 


.32038 









4 

1 

.42792 

.20168 

.13444 

.10698 








2 

1 

.25347 

.16898 








5 

1 

.41156 

.19167 

.12579 

.09605 

.08231 







2 

1 

.22368 

.14679 

.11208 








3 



.19221 








6 

1 

.40072 

.18667 

.12105 

.09080 

.07464 

.06679 






2 


.20861 

.13529 

.101471 

.08341 







3 



.16457 

.12343 







7 

1 

.37715 

.17527 

.11304 

.08394 

.06782 

.05842 

.05388 





2 


.19004 

.12258 

.09103 

.07354 

.06335 






3 



.14232 

.10569 

.08538 







4 




.13731 







8 

1 

.39389 

.18276 

.11746 

.08669 

.06935 

.05873 

.05221 

.04924 




2 


.19382 

.12458 

.09194 

.07355 

.06229 

.05538 





3 



.14011 

.10341 

.08272 

.07005 






4 




.12211 

.09769 






9 

1 

.39286 

.18226 

.11881 

.08591 

.06829 

.05727 

.05092 

.04556 

.04367 



2 


.19019 

.12398 

.08965 

.07126 

.05977 

.05313 

.04754 




3 



.13855 

.10019 

.07963 

.06678 

.05938 





4 




.11265 

.08958 

.07512 






5 





.10680 






10 

1 

.39373 

.18242 

.11677 

.08560 

.06775 

.05646 

.04891 

.04379 

.04054 

.03937 


2 


.18784 

.12024 

.08813 

.06977 

.05814 

.05036 

.04508 

.04174 



3 



.12988 

.09520 

.07536 

.06280 

.05440 

.04871 




4 




.10633 

.08417 

.07014 

.06076 





5 





.09716 

.08098 






The evaluation of the covariances was much more troublesome, requiring the 
evaluation of iterated integrals of the form 


rxf(x)F^(x) r - F(t)]*dtdx. 

J—oo 
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Necessary linear combinations of such forms give rise to considerable loss of 
accuracy. The covariances are believed to be correct to within 1 unit in the 
second decimal (except for one or two values which may be off by two units). 

TABLE VI 

Correlation coefficients X 10^ between order statistics x(i | n), x(j | n) for the 



Better tables of these covariances are badly needed, and it is hoped that someone 
will provide them. 

The asymptotic values are correct to the two decimals given. 
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6. Computation in terms of the representing function. It will prove con¬ 
venient in working with the special distribution, as indeed it does in many 
statistical procedures, to introduce the representing {unction r(u), which is a 
monotone function such that 

Pr {r(ui) <x< r{v^] = , t «2 > Ui • 

Thus if u has a uniform (= rectangular on [0, 1]) distribution then x = r(u) 
defines a variate with the given distribution. 

The zth order statistic of n from the uniform distribution, Ui\n , is distributed 
according to 

- «)*"* dM , 0 < « < 1 , 


where it is important to remember that ui,n is the largest and not the smallest 
order statistic; and the joint distribution of u = u,*,n and v = uy,n y{j>i)j 
is given by 

— i) r . . ? .1 dudv^ 0 < v < u < 1, 

L ^, j i^n j j 

where . . ? . is a multinomial coefficient. 

liyj - i,n 

The means, variances, and covariances which we desire can be written as 
follows (it is immaterial whether we think of expectations over x’s or over u^s ): 

E(x„n) = E{r(un„)) = i(^{j j[‘ - «)<** dv, 

var (a^t'in) = E{xiin) (^(xim)) = E(r (uim)) “• E(x4in) 

= t du - ( E ( xt , n ) f , 


COV (x»|n I ®/|n) — 

= E(r(ui,„)r(u„n)) - E(Xi,n)E(x,,„) 

= i(J - i) r . . ” „ .1 r f r(uMv)v’^’(u - vy-*-\l - du dv 

L*, J If n J J •'0 

- E(Xi,n)E(Xj,n) 


Introducing E,,t by 


E.,, = If r(u)r(v)u*v* du dv. 


we have 

_j] 
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and, in particular, 

E{Xix7,Xti2) = 2 ^ 0,0 9 

£^(xi,6^4i6) = 60J^2,i — 120J57i, 8 4" 60JS7o,* • 
Introducing E.,, by 

= f r^{u)u* du^ 


we have 

Eix \ n ) 

and, in particular, 
Introducing E^ by 


<0 ?<-'>*(*'I 

■^(^215) = 20^3,8 — 20J^4,4 . 



we have 

and, in particular, 

^(^8|6) ~ SOiSi — GOi^s + 30JB?4 • 


Thus the computation of the desired means, variances, and covariances is 
reduced to the computation of the integrals £7,, , and . 

We shall also want to calculate the asymptotic approximations to the means, 
variances, and covariances of the order statistics. For the uniform distribution, 
it is well known that 


mean (2/t|n) 


n — i + 1 
n + 1 


var 


i(ji — i + 1) 

{n + mn + 2 y 


COV (Ui\nUj\n) 


i(n — y + 1) 

(n + l)Kn + 2 )’ 


a < i). 


These asymptotic formulas are transformed from w to a; by the relations x = 
t{u) and dx = r\v) du, giving 


/ \ /n — i l\ 

approx mean (a;i|„) = r t — )» 

approx var (a:<|„) = ^r' ^ 


i(n — i + 1) 
(n+ l)Xn + 2)’ 
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approx COV {Xi\nXi\n) 



( n — t + l\ i + 1 

n+1 V (n + l)*(n + 2) ’ 


(S < 3)r 


as noted above, in our calculations we have replaced n + 2 by n in the denomi¬ 
nator. 


6. Reduction of integrals for the special case. When the reoresenting func¬ 
tion is 

we obtain a symmetrical distribution with long tails. (For the normal dis¬ 
tribution r{u) = o(ln u) as u —> 0). The integrals we want are 

= [' {(1 - u)-'^ - u-^\u*du, 

Jo 

E.,. = {(1 - u)-^ - du, 

E.,, = jj' JT* {(I _ n)-^ - {(1 - du dv, 


which can be expressed as 

E. = AM - BMf 
E.,, = A,,M - 25...(X) + 

E,,t = ^,.*(X) - 5.,,(X) - C.,M + DU^), 


where 


AM = f (1 — u) ^ u'’ du = 6(--X, s), 

Jo 

pi 1 

B,(X) = / u~^u‘ du = ——-- , 

(1 - = 6(-2X, s), 

B, .(X) = f\l - u)~^u~^u‘ du = 6(-X, s - X), 

JO 

C. ..(X) = I du = 

(1 — uy^(l - v)'^u’v* du dv 
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_ V /A /—V’ + 1 — 2X, 0 

S V7 ^ ^ t + 1 - X ’ 

J?.,,(X) = f r (1 - uy^v^u‘v*dudv 
Jo Jit * 

_ M . W 6(t + 1 - X, < - X) _ 6(s + i + 1 - X, -X) 
to Vi/ ^ i + 1 - X < + 1 - X 

C. ,,(X) = £“"^(1 - v)'^u‘v* du dv 

~ g -j- " i i- ~ x <) ~ ^(s + < + 1, —X, —x)|, 

D, ,,(X) = j j vT*'v~^ u‘v'du dv 

_1_ 

(^ + 1 — X) (s + < + 2 — 2X) ’ 

where throughout 


b(p, q) = 


p!g! 

(P + 8+1)! 


r(p + l)r(g + 1) 
r(p + 8 + 2) 


= B(p + 1, ? + 1). 


7. Calculations for the special distribution. The computations for the special 
distribution were made from the formulas in the preceding section.. The quan¬ 
tities tI, Bj C, D were computed from r = s = 0 to r+s = 8, whence the values 
of Eg , E„ J Ett were calculated. The values of the means, variances, and co- 
variances were then obtained from the formulas of section 3. 

The means, variances, and covariances are believed to be accurate to the five 
decimal places given. 


8. Formulas and accuracy for the uniform. The means, variances, and co- 
variances of the uniform are given near the end of section 5. Since r{u) = u, 
they are also the values given by the asymptotic approximation, when n + 2 
is used. 

The tabulated values were computed to six decimal places and rounded to the 
four or five decimals given. 
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SEQUENTIAL CONFIDENCE INTERVALS FOR THE MEAN OF A NORMAL 
DISTRIBUTION WITH KNOWN VARIANCE 

By Charles Stein and Abraham Wald 
Columbia Univeraity 

1. Stunmary. We consider sequential procedures for obtaining confidence 
intervals of prescribed length and confidence coefficient for the mean of a normal 
distribution with known variance. A procedure achieving these aims is called 
optimum if it minimizes the least upper bound (with respect to the mean) of the 
expected number of observations. The result proved is that the usual non¬ 
sequential procedure is optimum. 

2. Introduction. The problem of sequential confidence sets in general has 
been considered briefly by one of the authors [1]. Let {X<), (i = 1, 2, • • •), 
be a- sequence of random variables whose distribution is specified except for the 
value of a parameter 6 whose range is a space Q. Sequential confidence sets are 
determined by a rule as to when to stop sampling, together with a function of 
the sample whose value is one of a specified class of subsets of fi. The class of 
subsets is chosen in advance depending on the purpose of the estimation. For 
example, it may be the class of all intervals of prescribed length or the class of 
all sets whose diameter does not exceed a given value. It is required that the 
probability that this (random) set covers d should be greater than or equal to a 
specified confidence coefficient a for all 6. A procedure for finding sequential 
confidence intervals is considered optimum if it minimizes some specified function 
of the expected numbers of observations. Here this function is taken to be the 
least upper bound. In contrast with the result of this paper, a case where se¬ 
quential confidence intervals may have an advantage over non-sequential pro¬ 
cedures has been given by one of the authors [2]. The X^ are independently 
normally distributed with unknown mean and unknown variance, and the prob¬ 
lem is to find confidence intervals of fixed length for the unknown mean. As 
was first shown by Dantzig [3J this cannot be accomplished by a non-sequential 
procedure. Another case where this is tme is the problem of finding confidence 
intervals of the form (po, kpo) where A; is a specified number greater than 1, for 
the probability in a binomial distribution. 

Let {X<}, (i = 1, 2, • ••)> be independently normally distributed with un¬ 
known mean f and known variance <ri . It is desired to specify a sequential 
procedure for obtaining confidence intervals of fixed length I for the mean 
This is provided by a rule according to which at each stage of the experiment, 
after obtaining the first m observations Xi, • • • , for each integral value m, 
one makes one of the following decisions: 

a) Take an (m •+• l)st observation. 

b) Terminate the procedure and state that the mean lies in the interval 

427 
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(y — §Z, F -i- JO, where Y = , • • • , Xm), Sm being a measurable real¬ 

valued function. The serial number m of the observation on which the proce¬ 
dure terminates is, of course, a random variable and will be denoted by n. 

For any relation R the symbol P(R | f) will denote the probability that R 
holds when f is the true mean of Xi . The confidence coefficient of a sequential 
procedure S is defined by 


(1) a{S) = gJ.h. P(F - JZ < { < F + JM {)• 

Denote by rioiS) the maximum expected number of observations, i.e. 

(2) no(S) = l.u.b. E(n | f, S) 

where E(n | {, S) denotes the expected value of n when {is the true mean and the 
procedure is used. 

A procedure S will be considered optimum if, for all S' such that a{S') = 

(3) no(S) < no(S'). 

It will be shown that an optimum procedure S{vy c) can be obtained as follows: 

a) For all m < i^, a fixed positive integer, take another observation. 

b) For m = V, terminate the procedure if 


(4) 


> cc\ 


and let F = ■ SX<. (The inequality (4) is used merely as a device for fixing 

Vl 

the probability of taking v observations, this random event to be independent 
of whether (F — JZ, F + JZ) covers {, given v.) 

c) Otherwise take a {v + l)st observation, terminating the process, and let 


F = 


+ 1 


y-4-l 


When c = 0, this is the usual non-sequential procedure. 

Clearly, 

(5) afS(v, c)] = P[xU > c}H + [1 - P{x?-, > c]]H , 

where 


Also 

(7) no[S(v, c)] = V + 1 — P{xl-i > c), 

By a proper choice of v and c we cq-n achieve any desired confidence coefficient 
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a > H ^ . There is no essential loss of generality in considering only the 

case = 1, and this will be done in the remainder of this paper. 

3. A lower bound for no(S) and an upper bound for a{S), Consider any 
sequential procedure S for obtaining confidence intervals of length L Put 

(8) a({, ® = P{F - iZ < f < r + i/1 «}. 

That is, a(f, S) is the probability that the confidence interval will cover the true 
mean i when the procedure S is used. According to (1), 

(9) a{S) = g.l.b. a({, S). 

In order to obtain a lower bound for noiS) and an upper bound for a(S), we 
suppose that the procedure S is applied when { is not a fixed number but a ran¬ 
dom variable normally distributed with mean 0 and variance Then the 
probability that the confidence interval covers f is 

(10) a(cr, S) = a({, S) ^ a(S) 

and the expected number of observations is 

(11) Sin I <r, S) = £“ Ein | S) rff ^ rtoiS). 

Let pmii, S), {m = 1, 2, • • • , ad. inf.), denote the probability that n = m 
when {is the true mean and procedure S is used. Put 

(12) p„(<r, S) = S) di. 

Since 

00 

(13) Sin I <r, (S) = 'El mpn,i<r, S) 

tnmml 

we obtain from (11) 

(14) 2 rnpmicr, S) ^ no(S). 

fn«il 

We shall now derive an upper bound for a{<r, S ). Since = { + c< where the 
€*• are independently normally distributed with mean 0 and variance 1, the joint 
distribution of { and Xi , (t = 1, • • • , m), is a multivariate normal distribution 
with 


( 15 ) 


Ei ^ EXi^O 
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and covariance matrix 


S ^ I 1 2 

a -t I c 


(16) E . - : 


2 2 . t 

<T CT -H 1 


2 2 
a a 


<r’ + lj 


Thus the conditional distribution of ( given , • • • , Xm is normal with mean 




r * I 1 2 

O’ + 1 O’ 

2 * I 1 

a a + 1 


(m — l)o’^ + 1 __ O’* 

mo^ + 1 ni<r^ + 1 

__ O’* (m — l)a* + 1 

07) = ,1) ^+1 m,r‘+l 


ff + 11 


+ 1 


mo’2 + 1 


mcr^ + 1 m<r^ + 1 


(m — l)o’* 4“ 1 
7710’^ 4“ 1 




and variance 


4 / ”* * 


' ' (mt< + 1)' 7 !».>■ + 1 

If Xi, * • • , X„i is a sequence for which the process is terminated on the mth 
trial, the conditional probability that the interval of length I will cover ( is 
clearly maximized by taking 

(19) v = 

mo*^ 4“ 1 1 

and, by (18) this probability has the value ff(Cm) where II is defined by (6) and 


^ - 4 /^ + 
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Hence, 

(21) a(,r,S) < '£pU<t,S)II{c„). 

From this and (10) we obtain 

(22) a(S) 

1 

This upper limit of a{S) and the lower limit of miS) given in (14) will be used 
later to prove that Siv,c) is an optimum procedure. 

00 

4. Maximum value of X Pm(<r, S)H(c,n) subject to the condition that 
1 

00 

Yl 'mpm{(Ty S) does not exceed a given bound. We shall show that the maximum 
1 

00 

of pfn{cr, AS)//(Cm) subject to 
1 

00 

E(n 1 o, 5) = 22 mp*(<r, S) g >> + o, 

1 

where v is a positive integer and 0 < a < 1, is obtained by choosing pm(<r, S) = 
pZ defined by 

pZ = 0 for m < V or m > y + 1 

(23) p? = 1 - a 

* 

Pp+1 = a. 

For, suppose to the contrary that there exists a sequence {pm] such that the 
following conditions hold: 

00 

P- > 0, 22 Pm = 1 

1 

00 00 

(24) 22 mprn < »> + o = 22 

1 1 

t,Pn.H(c„) > t.plHic„). 

1 1 

I 

We have 

(25) = 

Put 

C = HM - me,) = dy. 

V 2t 


( 26 ) 
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With the aid of = 1 — Pm , we obtain from the last two inequalities 
in (24) 

(27) 0 < E (p„ - pZ)H(c„) -CE(p„,- pl)m = E (p». - vlWn. 

1 1 mptfv 

where 


(28) Km = H{cm) - H{cp) - (m - v)[HM - //(c.)]. 

Clearly Kp^i — 0. Also, for m < v, since the integrand is a strictly decreasing 
function of y, 


(29) 


y~^e~^''dy — / 

•> C % 


dy 


< (» — w) ^ p ‘c"*" —(y — m) \ y * t 


lv-c| 


= 0 . 


Similarly for m > v + 1 , Km < 0. But = 0 for m 5 *^ v, + 1 so that 
(30) E (Pm - tZ)K^ < 0 

mfip,iv+l 


which contradicts (27) since Kp+i — 0. 

Thus, we have shown that the inequality 

(31) Ein 1 <r, S) < » + o 
implies the inequality 

(32) E P>n(<r, S)H(,c„) < (1 - o)//(c,) + aH(cr+i). 

I 

6. Proof that S{v, c) is an optimum procedure. Since, according to (14) 
and (22) 

(33) no{S) > E(n | <r, S) and a(S) < E Pm(<T, S)H(c„), 

1 

it follows from the result expressed in (31) and (32) that, for any procedure S 
satisfying the inequality 

(34) noiS) < V + a, 


we must have 


(36) a(S) < (1 — a)H{Cp) + oJT(c,+i) 

identically in <r. Since is continuous, it follows that 

(36) a{S) < (1 - a)H[y~v I) + aH^VT+l 

for any procedure S satisfying (34). 
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The right hand side of (36) is a[Sivf c)] where c is chosen so that 

(37) 1 - a = PIxLi > c(. 

We use an indirect proof to show that S(v, c) is an optimum procedure. Sup¬ 
pose to the contrary that there is a procedure S' such that 

(38) a(S') = a[Siv, c)] 
but 


(39) noiS') < no[5(v, c)]. 

By (5) and (7), a[S(y, c)] is a continuous strictly increasing function of 

+ 1 — P{xv-i > c] 

and this latter is no[S(p, c)]. If we choose /, c' so that 
no(S') <p' + 1- P{x5-i > c'l 


(40) 


< p + I -P{xLi > cl, 


it follows that 


(41) a[S(/,c')] <a[S(v,c)] -a(S0. 

But (41) andjthe first part of (40) contradict the result expressed in (34) and 
(36). 
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NOTES 

This section is devoted to hnef research and expository articles on methodology 
and other short items, 

A USEFUL CONVERGENCE THEOREM FOR 
PROBABILITY DISTRIBUTIONS 

By Henry Scheff6 

University of California at Los Angeles 

In problems of establishing limiting distributions it is often apparent that the 
probability density pn{x) of a random variable Xn has a limit p{x) ; throughout 
this paper n = 1, 2, 3, • • • , and all limits are taken as n —► qo . If p{x) is the 
density of a random variable Xy what we really care about then is whether the 
limits apply to probabilities, which involve integrals of the densities: Does 
lim Pr{Xn in 5} - Pr{X in S} for all^ Borel sets S, or, does 

(1) lim / pn(x) dx= p(x) dx ? 

The question is thus one of taking a limit under an integral sign. Perhaps the 
most widely used justification of such a process is the following theorem of 
Lebesgue [1, p. 47; 2, p. 29]: If for a sequence \fn{x)} of integrable functions, 
lim fn{x) = f{x) for almost all x in Sy then a sufficient condition that 

lim / fn{x)dx = / fix) dx 

Js Js 

is that there exist an integrable function gix) which uniformly dominates the 

sequence {/^(x)}, that is, |/n(x) | < gix) for all n and all x in S, and / gix) dx <oo. 

Js 

For example, in the excellent new treatise by Cramer the limitij g form of the 
^distribution is treated as follows [1, p. 252; other examples < n pp. 369, 
371]: For n degrees of freedom the ^-variable has the density 

(2) Pn(x) = c»(l + 

where 

(3) c„ = (nir)"*r(Kn + l))/r(in). 

It is shown fairly easily that lim pnix) = p(x), the density of ^^(0,' 1), where 

^ In defining the convergence of a sequence of distributions to the distribution of a dis¬ 
continuous random variable X it is desirable to modify this requirement so that it is de¬ 
manded only of sets S which are continuity intervals of X [1, p. 83]. We are concerned here 
however only with the ^‘absolutely continuous case'’ where X has a probability density p(x ). 
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Ninij (T^) denotes the normal distribution with mean m and variance 
to prove 


lim 



dx 



2 

<7 . 


Then 


Cramer shows that {p»(x)) is uniformly dominated by an integrable function. 
It is instructive to consider some examples where 



does not equal 

(5) f lim pn(x) dx, 

J—00 


In the examples (t), (m), (m), lim pn(x) = 0 for all x and hence (5) is zero for 
all £. 

(0 Pn(x) = IfoT —n — 1 < X K—Uy zero elsewhere. Then (4) equals 1 for all {. 

(m) Pn(x) = 1/n for —in<x< Jn, zero elsewhere. Here (4) equals Jfor all 

f. 

(in) pn(x) = 2n^x for 0 < x < llriy zero elsewhere. Now (4) is zero for 
{ < 0, unity for f > 0. 

An example in which lim Pn(x) 0 is 

(iv) Pn{x) ~ i[hn(x) + po(x)], whcrc hn is the pn of one of the above examples 
and po is a fixed density. Then lim Pn(x) = ipoix). Now (4) exceeds (5) by 
half the amount it did in the corresponding above example. 

The essential features of these examples could be obtained with normal 
distributions but would involve a little more computation, for instance, N(—n,l)y 
N(0y n^), N{l/ny 1/n^), for examples (t), (w), (in), respectively. 

We note that in none of these examples is lim Pn(x) a density. This suggests 
that the trouble might perhaps be prevented by requiring that lim Pn(x) be a 
density—^which happens in the case from which we started. This surmise is 
correct. We may formalize the situation as follows: 

Definition. A function f(x) mil he called a density if it is non-negative and 

J f(x) dx = 1, Here R denotes the whole space of x. 

R 

The reader may think of a univariate density, where a; is a read variable and 
R is the real axis, but theorem and proof run the same for a fc-variate density, 
where a; is a point in a A:-dimensional Euclidean space R, 

Theorem^. If for a sequence {pn(x) } of densities 

lim pn(x) = p(x) 


* The hypotheses of this theorem, while perfectly adapted to applications in probability 
and statistics, would not seem the “natural’^ ones in real variable or measure theory. Pro¬ 
fessor A. P. Morse has remarked to the writer that, if the theorem has not been stated in this 
form before, it is at least an easy corollary of some more general results known in that field. 
Nevertheless our direct proof based only on the familiar Lebesgue theorem and using only 
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for almost all x in R, then a sufficient condition that 

lim / pn(x) dx = p(x) dx, 

Js Js 

uniformly for all Borel sets S in R, is that p{x) he a density. 
Proof. Let us write the difference 

( 6 ) Pn{x) - p{x) = 8n{x). 

Then 


(7) 


8n(x) 0 


for almost all x in R. Also 
(8) [ 5ndx 

and so it suffices to prove that 
henceforth denotes a Borel set. 


= Pn dx — p dx, 

Js Js 

/ da: 0 uniformly for all S in R, where S 

Js 

If in (8) we let S = R we get 


(9) f Sndx =0 

Jr 

since p„ and p are densities. We now split the difference 5„(a:) into its positive 
and negative parts: Let 

(10) = K«n + 1 S« 1 ), «; = iiSn - I «» I ), 

SO that 

+ S~, dt >0, s-< 0. 

Froih (7) and (10), we find 

( 11 ) 6 ; 0 
for almost all x in R, and from (9), 

(12) f Stdx+ f S-dx =0. 

Jr Jr 


very simple manipulations may be of interest to readers of the Annals. Professor Morse 

also pointed out that the stronger result lim f | p»(x) — p( 2 ) | dz ■■ 0 uniformly for all 8, 

Ja 

may be stated. This follows from our proof since 


I \ pn - p\dx f in dx 

Ja Ja 



dx. 
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By virtue of (6),«, > -p. Nowifi, < 0,«I = «„ > -p, and if i, > 0, ilT = 

0 > — p, and hence in every case 0 > 61 > -p. Since we now have ] 1 < 

p(x) and / p(x) dx = 1, we may apply’ the Lebesgue theorem to get 
Jr 

lim / 8ndx== / lim Bn dx. 

Jr Jr 

The right member is zero because of (11). It then follows from (12) that 

lim / Bt dx is also zero. The relations 
Jr 

0 < f Btdx < f Btdx-^0, 

Js Jr 

0 > f Bn'dx > f Bndx->0 

Js Jr 

guarantee that the quantities St dx and / Bn dx have the limit zero uniformly 

^ Js Js 

for all S, and hence the same is true of their sum (8). 

Returning to the example (2), we remark that it is practically obvious that the 
second factor on the right has the limit but it is not quite so obvious that 
lim Cn = (2^^)“^ This situation is typical of many applications where it is 
more difficult to evaluate the limit of “the’’ constant than the limit of the re¬ 
maining factors, and one wonders after obtaining the latter limit whether the 
constant is not automatically forced toward the limit desired for it, and whether 
the dii ect calculation of its limit could not be avoided. Let us put the question 
as follows: Suppose that 

{PnM = CrJnM} 

is a sequence of densities and that 

p(x) = cf(x) 

is also a density. Then if lim fn(x) = f(x) for almost all x, may we conclude 
that lim Cn = c? If so, we could then apply the above theorem without having 
evaluated the limit of the constant or produced a dominating function. Un¬ 
fortunately the answer to this question is no, as shown by example (tv) above: 

* Although our proof rests on the Lebesgue convergence theorem, this theorem is applied 
nto 5(a;) and not to Pn(x). While in most cases of practical interest the sequence |pa(x)| 
is uniformly dominated by an integrable function, it is possible to devise a simple example 
where this is not true and yet our theorem applies: Let p»(») 1 for l/(n -b 1) < x < 1 

n 

and for o» < rc < a»+i, zero elswhere, where 2 1/i. Then sup pn(x) « 1 for 

t-ii 

all X > 0, nevertheless lim Pt,(x) is a density, namely that of the uniform distribution on 
( 0 , 1 ). 



438 


M. KAC AND A. J. F. SIEGEUT 


If we let /n(a;) = Kix) + poW, and /(x) = poW, then lim /n(x) = /(x), but 
Cn = J and c = 1, hence lim Cn c. Employing the assumption that pn(x) 
and p(x) are densities we see 

Vcn = f fnCr) (lx, lie 

, Jr 

and hence lim c„ = c if and only if 

(13) lim f fn{x) (lx = f 

Jr Jr 

It follows that in such cases if we wish to establish a limiting distribution in the 
sense (1), we may cither prove lim c„ = c, or we may justify (13), say by produ¬ 
cing a suitable dominating function, but we need not do both. No doubt the 
first alternative would be preferable at all but the most advanced levels of 
teaching or exposition. 


= [ fix) dx, 
Jr 

lim / n ( x ) dx. 
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AN EXPLICIT REPRESENTATION OF A STATIONARY 
GAUSSIAN PROCESS 

By M. Kac^ and A. J. F. Siegkrt 

Cornell University and Syracuse University 

1. In a paper which will soon appear in the Journal of Applied Physics [1] 
the authors have introduced methods of calculating certain probability dis¬ 
tributions whi(^h are of importance in the theory of random noise in radio re¬ 
ceivers. 

The complexity of the physical problem and occasional uses of heuristic reason¬ 
ings may have obscured some of the mathematical points. For this reason the 
authors felt that it may be worth while to illustrate one of the basic ideas on a 
simple but important example. 

2. A stationary Gaussian process is a one parameter family x(0 of random 
variables such that: 

(a) . x(0 is normally distributed; the mean and the variance being inde¬ 
pendent of t 

(b) . the joint probability distribution of x(<i), x(fe), • • • , x(/r) is multivariate 
Gaussian whose parameters depend only on the differences tj — 4 . 


‘ John Simon Guggenheim Memorial Fellow. 
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We assume, for the sake of simplicity, that the process is normalized, i.e., 
j&fxW} = 0, E{x\t)} = 1 

and we define the correlation function p(r) by the usual formula 

p(t) = E{x{t)x(t + t)}. 

It is then well known* that a distribution function (t{u) exists such that for all r 


( 1 ) 


p(t) = I COS ur da (u), 
J—00 


3. Let 0 < 8, t < T and consider the symmetric kernel 

K(s, t) = p(s - t). 

The fact that <r(u) is non-decreasing implies that the kernel p(s — 0 is quasi- 
definite, i.e., for every L* function g{f) on (0, T) one has 


f f g(s)p(s - t)g(t) ds dt > 0. 
Jq Jq 


Thus the eigenvalues of the integral equation 

( 2 ) r pis - t)f(i)dt = \fis) 

Jo 

are non-negative. Moreover, denoting by X,- the eigenvalues and by fj{i) the 
corresponding normalized eigenfunctions of (2) we have by the classical theorem 
of Mercer (see [4], in particular part 6 of Ch. I) that 

(3) p(s -0=2 Xy/j(s^/j(0, 

J 

where the series on the right is absolutely and uniformly convergent. It should 
be noted that in virtue of (1) p(t) is a continuous function. 

4. Let now Gi, Gz, Gs, • • • be independent, normally distributed random 
variables each having mean 0 and variance 1. 

Consider the series 

(4) 2 VWy(0. 

} 

Since for each t we have 

2 {VKi/iur = i: xy^(o = p(o) = 1, 

3 i 

we infer that for each t the series (4) converges in the mean to a random variable 
xit). Moreover, by a theorem of Kolmogoroff [6], the series (4) converges, for 
each t, to xit) with probability 1. 


* See [2]. The theorem in question (in a somewhat different form) seems to have been 
first established by N. Wiener in [3]. 
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Thus we may write 

(5) .r(0 = E V\G,f,{t). 

It is now easy to show that x(t) thus defined is a stationary Gaussian process 
in (0, T) with the correlation function p(r). 

In fact, 

^;|u,(s)a.(/)} = Xj/;(«)/,W = p(s - t), < S, I < T, 

1 

and conditions (a) and (b) of section 2 follow from the well known properties of 
linear combinations of independent Gaussian random variables. Of course, 
we are dealing here with infinite linear combinations but the mean convergence 
noted above, is sufficient to justify the extension to our case. 

6. It is more illuminating to think of the random variables G, as measurable 
functions G,(c«)) defined on an abstract set in which a Lebesgue measure has 
been established (the measure of the whole space being 1). 

The representation (5) can then be written in the equivalent form 

(0) - E 

J 

The equality, as established in section 4, holds for every t in the sense of mean 
convergence. Moreover, by the theorem of Kolmogoroff cited above, and by 
Fubini’s theorem the equality (6) holds for almost every pair (t, w), (0 < i < T), 
in the sense of ordinary convergence. 

Furthermore by Mercer‘s theorem (remember that X| > 0) 

pT 

\ = / p{s — s) (is = T 

j Jo 

and hence 

E = E X, f = E X, = r < 00. 

3 i JQ 7 

Thus 

\G]{o3) 

converges for almost every « and therefore the series 

(7) E V\GMf,(t) 

3 

converges in the mean for almost every w. 

Combining this fact with the observation that (7) converges almost every¬ 
where to x{t, «) we see that, for almost every w, the series (7) converges in 
the mean to x{ty co) and that consequently 
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( 8 ) f\\t,^)dt = 

JO j 

for almost every w. 

It should be noted that (8) could not, in general, be derived by just appealing 
to Parseval’s relation. The main reason is that Parseval’s relation holds only 
for complete orthonormal systems whereas the orthonormal system {/n(0) of 
eigenfunctions may fail to be complete. If the kernel p(s — t) is positive- 
definite (in which case all the eigenvalues are positive instead of just non-nega¬ 
tive) then it is known that the eigenfunctions form a complete set. This actu¬ 
ally, happens to be the case in most physical applications. 

6. An important application of (8) is the calculation of the characteristic 
function of the distribution function of the random variable 

(9) / = f x^{t, w) (it. 

Jo 

In fact, 

(10) E{exp HiD } = n ^^{exp = 11(1 - 

i i 

The probability density of / is the Fourier integral 

h- L„ n (1 - 

which, unfortunately, in most cases cannot be calculated explicitely. If 

P(r) = e-"'''. 

in which case the process is also Markoffian, the eigenvalues X,- can be cal¬ 
culated explicitly^ but in more complicated cases it is quite difficult to deter¬ 
mine them. 

7. If p(r) is absolutely integrable and (r(p) absolutely continuous then, setting 

A{u) = <r'(u), 

we have A{u) >0 and 

p(t) =‘ r coa utA(u) du = r e*'‘'B{u) du, B{u) 

J—OO J—00 

* See lOJ, in particular section 4. We take this opportunity to correct two misprints in 
this note. In the last formula on p, 64 M should be replaced by N. Also the limits of 
integration in formula (6) should be 0, « and s, p -f- 9 instead of 0, p 4" $ and 0, p -f- 

The N.D.R.C. Report 14-305 to which a reference is made has been declassified in the 
meantime. It contains results which originated both [1] and the present note. 

* These and related results were stated in the abstract [71 by M. Kac. The paper is now 
being prepared for publication. 
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It can then be shown^ that 

i = ^TT f B^{u) du = f p^(t) dr 

T—*CO 1 ] V—00 V—00 

and 

bm ~ 23 ^j = (2^)^ f B^(u) du. 

T—*00 1 j J—eo 

It follows now by standard methods that the characteristic function of 



approaches, as T —> oo , 

where 

a* = r p\t) dr. 

•t-oO 

Thus, as T —> <», the distribution of (11) becomes normal with mean 0 and 
variance <r*. 
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APPROXIMATE FORMULAS FOR THE RADII OF CIRCLES 
WHICH INCLUDE A SPECIFIED FRACTION OF A 
NORMAL BIVARIATE DISTRIBUTION 

By E. N. Oberg 
University of Iowa 

1. Introduction. Given the normal bivariate error distribution 

( 1 ) 

The purpose of this paper is to present certain approximate formulas for the 
radii of circles whose centers are at the origin, which include a prescribed pro¬ 
portion, p, of errors. The formulas are, for given cr* , o-y , and p, 
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(2) 

Ri = \/2o-*<ry In (1/[1 — p]) 

(3) 

= y/(<rl + <rl) Jn (l/ll - p]} 

and 


(4) 

/?3 = ((Tx + OV(l/2) In (1/[1 — p]). 


In section 3 we present tables of p', the true proportion of errors contained in 
circles whose radii are given by the above formulas. These tables reflect the 
goodness of approximation of each formula to the true radius, R, for 0.1 g p g 
0.9 and 0.5 ^ o-x/o-y ^ 0.9. Also, a brief statement is included for the same range 
of p but with 0.1 ^ (Tx/cy ^ • 4. 

2. The derivation of the formulas. The proportion p of errors that fall 
within an area A on the o^p-plane is given by 

(5) p = j^<p(x,y) dA. 

If the area is bounded by any member of the family of elipses 

2/2 , .. 2/2 x 2 

X /(Tz + y /(Ty = \ y 

the above integral may be evaluated directly. The result is 

p = 1 - e , 

whence 

= 21n(l/[l ^ p]). 

Thus the ellipse with semi-axes 

(6) <rzV2 In (1/[1 - p]), <ry\/2 In (1/[1 - p]), 

measured from the origin along the x and y axes respectively, will include ex¬ 
actly the prescribed proportion of errors. 

Frequently, however, it is desired to know which circles rather than which 
ellipses include a certain proportion of the errors. In this case it becomes 
difficult to obtain a formula for the true radius from (6) unless <r* = ay in which 
case R is given by either one of the formulas in (6). However, a natural ap¬ 
proximation to make is to equate the area of a circle of radius, say Rj to the area 
of the ellipse whose semi-axes are given in (6). This gives formula (2), 

Ri = y/2ag(ry In (1/[1 — p]), 

which can be expected to give a fairly close approximation to true R if o** is 
close to ay . If cr, <ry , it has been shown that this formula underestimates 
true R which is undesirable in some applications [1]. That is, if is used to 
estimate, say the radius of a circle to include 50% of the errors (p = .5), it will 
give a value which includes less than the desired proportion. The first table in 
the last section gives a numerical verification of this fact. 
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To obtain formula (3) we consider formula (5) when A is a circle of radius R. 
We have 

p = 4 / / y) dy dx, 

Jo Jo 

By making the transformation x = <r*r cos d,y = ayV sin 0, and by carrying out 
the integration with respect to r the above formula becomes 

p = 1 - {2/t) f" dd. 

Jo 


We let 


and 


a = Ry(<rl + 4), al)/(al + al), 

O’*/ O’y ^ J (T* ^ (Ty • 


Then 

a = R^/alil + €^), and /3 = (1 — €*)/(! + e^), which is less than unity. 
This substitution will be helpful later in preparing tables. The fact that o-* 
is taken less than Cy places no limitation on the final results since we only have 
to interchange axes in the other case. The above integral may now be written 

as 


/•t/2 


a) 


= 1 - (2/«-)«““ / e 
Jo 


—a^/(l—j8co82®) 


de 


-a/Jco 8 29/ (1 -^co 8 2^) 


de. 


The integrand, say F($), in the last integral of (7) can be shown to be monotone 
increasing from to as B varies from 0 to x/2. Furthermore, it crosses 

the line F{d) = 1 somewhere in this interval and differs but little from it any¬ 
where if the ratio ax/cry is close to 1, since /3 is then close to zero. If, therefore, 
we replace the integrand by F($) = 1, we have p = 1 — e"®. Hence, if a is 
replaced by R^/(al + o-J) and the result solved for R, we have formula (3), 


R 2 = V(cl + cl) In (1/[1 - p]). 

Finally, formula (4), 

Rs = {<^x + OV(i) In (1/[1 — pj), 

is obtained by taking the root-mean-square of the former two. This formula 
has certain advantages over the other two, the most obvious being that <r» and 
ay enter linearly so that it Is simple to evaluate for given ax , ay , and p. Sec¬ 
ondly it will be seen by the tables and additional comments made in the last 
section that when p = 0.5,^ Rz overestimates tnie by a slight amount for all 


‘This particular value of p gives the circular probable error. In this case Rt ~ 
0.fi887(a. + c). 
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values of o-x/cry, and it gives a fairly close approximation to true R for all p 
when (Tx/oTy ^ 0.5. 

We close this section by making a few brief comments. In the first place, 
if a ny of the ab ove formulas is to be computed from a sample of data, we take 
V^2JxV(n — 1) and \/st/V(n ~ 1) as estimates of cr* and <Ty respectively. Fur¬ 


thermore, we test the significance of these statistics by known formulas [2]. 
Finally, ax and ay may be replaced by /|^/^ Dx and where Dx is the 

population mean deviation. Thus, for example, 


/?3 = {Dx + Dy) 


i/f'”' 


3. Tables. The first formula in (7) is useful in testing by means of numerical 
integration the goodness of approximation of the formulas Ri, R 2 f and Rz to 


TABLE I 


p computed by means of formula Ri 


\ 

.1 

.2 

1 

.25 

,3 

.4 

.5 

.6 

.7 

.75 

.8 i 

.9 

.5 

.0988 

.1951 

.2425 

.2893 

.3815 

.4720 

.5615 

.6508 

.6960 

.7422 

.8408 

.6 

.0944 

. 1974 

.2459 

.2942 

.3899 

.4846 

.5786 

.6726 

.7198 

.7676 

.8668 

.7 

.0997 

.1987 

.2480 

.2972 

.3950 

.4924 

.5894 

.6864 

.7350 

.7838 

.8835 

.8 

.0999 

.1995 

.2492 

.2989 

.3981 

.4970 

.5958 

.6946 

.7440 

.7936 

.8935 

.9 

.1000 

.1999 

.2498 

.2997 

.3996 

.4993 

.5991 

.6988 

.7483 

.7986 

.8985 

1.0 

.1000 

.2000 

.2500 

.3000 

.4000 

.5000 

.6000 

.7000 

.7500 

.8000 

.9000 


the true value of R. We construct the tables by replacing in a by one of these 
formulas, say formula Ri . This gives a = [2 € /(I + c^)][l/(l -- p)]. Since 
jS = (1 — c*)/(l + e^), the right hand side of the formula in (7) may then be 
evaluated for a choice of € and p giving a value we denote by p'. This is the 
actual proportion of errors that is included in the circle whose radius is Ri . 
If Ri gave true R, then p' would be equal to p, so we may regard the difference 
of p and p' as a measure of the error arising w’^hen Ri is used to estimate R. 

In the following tables the chosen values of p and € = ax/ay are listed in the 
first row and column respectively. The remainder of the tables include the 
corresponding values of p'. 

We also have computed tables for 0.1 ^ ax/ay g 0.4 which Ave have not in¬ 
cluded in this paper since for this range of values of ax/ay , all of the formulas 
give approximations that depart considerably from true R except Rz when p = 
0.5. For this case, p' = .4776, .5004, .5109, and .5120 when ax/ay = 0.1, 0.2, 
0.3, and 0.4 respectively. 
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The difference between an entry in a column and the corresponding value 
of p at the head of the column reflects the error in estimating true R by means of 
i?i, /^ 2 , and Rz . For example, if p is chosen as .5 and o-^/o-y = .7 then Rz 
gives the radius of a circle which includes 50.13% of the errors. Thus Rz 
overestimates true R by including .13% more of the errors. 

By examining the tables it is seen that when 0.1 ^ p ^ 0.3, Ri gives the best 
approximation to the true value of 7?, ^vhile Ro gives the poorest. If 0.4 ^ p ^ 

TABLE 11 


p computed hy means of formula R 2 


, / 

/ 

.1 

i 

i 

.2 

.25 

.3 

.4 

.5 

.6 

.7 

.75 

.8 

.9 

.5 

.1215 

.2363 

.2012 

.3440 

.4467 

.5432 

.6346 

.7217 

.7641 

.8060 

.8907 

.6 

.1116 

.2202 

.2732 

.3255 

.4274 

.5261 

.0218 

.7146 

1.7600 

.8050 

.8949 

.7 

.1057 

.2100 

.2616 

.3127 

.4140 

.5136 

.6116 

.7081 

.7558 

.8032 

.8976 

.8 

. 1022 

.2030 

.2540 

.3051 

.4056 

[.5055 

.6048 

.7034 

.7525 

.8014 

.8991 

.9 

.1005 

.2000 

.25091 

.3012 

.4013i 

.5012 ;6011 

.7(K)8 

.7506 

.8003 

.8999 

1.0 

.1000 

.2000 

.2500 

.3000 

.4000 

.50001.6000 

.7000 

.7500 

.8000 

.9000 


TABLE III 

p' computed hy means of formula Rz 


\ 

X 

.1 

.2 

.25 

.3 

.4 

•5 

.6 


.75 

.8 

.9 

.5 

.1102 

.2161 

.2674 

.3176 

.4152 

.5092 

.6001 

.6887 

.7327 

.7768 

.8694 

.6 

.1056 

.2089 

.2597 

.3100 

.4090 

.5059 

.6009 

.6944 

.7408 

.7872 

.8817 

.7 

.1027 

.2044 

.2548 

.3050 

.4046 

.5031 

.6007 

.6974 

.7456 

.7937 

.8908 

.8 

.1011| 

.2017 

.2519 

.3020 

.4018 

.5013 

.6003 

.6991 

.7483 

.7976 

.8963 

.9 

1.1003 

.2004 

.2504 

.3004 

.4004 

.5003 

.r)(X)l 

.6998 

.7496 

.7995 

.8992 

1.0 

.1000 

.2000 

.2500 

.3000 

.4000 

.5000 

.6000 

.7000 

.7500 

.8000 

.9000 


0.75, Rz gives the best and Rz the poorest; and if 0.8 ^ p ^ 0.9 Rz gives the best 
and Ri the poorest. Thus formula Rz for general use gives the best overall 
approximation. It may be remarked at this point that bounds for the true 
value of R can be found by applying two of the formulas, one of which over¬ 
estimates while the other underestimates R. From the tables it is apparent that 
this can be done for values of p ^ 0.8. 

Finally, these formulas may be used to test roughly the normality of the data. 
For example, if proper estimates^ of o-, and cry are made from the data, and the 


* See section 2. 
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corresponding value of Rz computed for a chosen p, then approximately, the 
proportion p' of plotted errors should fall within the circle of radius Rz, 

REFERENCES 
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A NOTE ON THE EFFICIENCY OF THE WALD SEQUENTIAL TEST 

By Edward Paulson 


Institute of Statistics^ University of North Carolina 


The sequential likelihood ratio test of Wald for testing the hypothesis Hq 
that the probability density function is/(X, 0o) against the one-sided alternative 
Hi that the function is/(X, ^i) has been shown [1] to have the optimum property 
of minimizing the expected number of observations at the two points 0 = 0o 
and 0 = 01 . Tables showing the actual magnitude of the percentage saving 
of this sequential procedure compared with the classical “best^^ non-sequential 
test have been calculated (see [1], page 147) for the normal case when 


fix, d) 




-(X - ey 
2 


In this note we will show that when 0i is close to do , the percentage saving is 
independent of the particular function /(X, 6) and the particular values 6i 
and 00 , so that the tables mentioned above can be used to show the percentage 
saving for any one-sided sequential test involving a single parameter, provided 
/(Z, 0) satisfies some weak restrictions. 

Let f{X, 0) be the probability density function of a random variable. Let 
Ei(n) denote the expected value (when 0 = 0,) of the number of independent 
observations required by the Wald sequential procedure to test the hypothesis 
Ho that d — do against 0 = 0i = 0o + A with probabilities a of rejecting Ho 
when 0 = 00 and |8 of accepting Ho when 0 = 0i. Let N be the number of in¬ 
dependent observations required to achieve the same probabilities a and 
for testing the hypothesis 0 = 0o against 0 = 0i by the most powerful non¬ 
sequential test. Let Ua and U^ be defined by the relations 



and 
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We will prove the following theorem: 

(aw) + 

\n}~ (c/a + U;>y 

provided f{X, 6) satisfies (he foUomng conditions: 

U) / /(X, 0) dx can he differentiated twice under the integral sign with respect 
to d. 

(B) All four of the integrals 


are continuous functions of 6* at $* = $o. A sufficient condition for {B) is that 
all the integrals he uniformly convergent with respect to 6* in some interval < 
6* < do + A, and all the integrands he continuous functions of X and 6*, A 

similar theorem holds regarding the limit of 

A-»0 

The proof is as follows: From [1], we know that 



Eo (n) = 


E«(z) 


+ 0 ( 1 ), 


where 


and o(l) —► 0 as A 0. 



Now 

EM - £[iog (|;^)]/(», *. 

= f llog/(x, $0 + A)]/(x, $t) dx - f [log/(x, 9o)]/(x, 9o) dx. 

J—ea J— M 
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Expanding log /(x, do + A) in a Taylor series about A = 0, we have 


log fix, 9, + A) = log/(x. Co) + A 


+ |«i, 
Mo I 


where 


and 


Oo < e* < So + A, r = , r = , 


I 

H 


From assumption (A) we find that 

f oo -00 

y{x, do) dx — 0 and i /"(x, (?o) dx = 0, 

00 J—eo 

while from assumption (B) 

f Rif{x, 0o) dx —► 0 as A —> 0. 

J—ao 

Therefore 

- -f[£[ 4 TL 

To find N for the most powerful non-sequential test, we make use of the fact 
(see [2]) that an asymptotically most powerful test for one-sided alternatives is 
given by a region of the type 

VNhfM) 

When A —> 0, A —> oo, and since is the sum of N independent variates with 

l^j _ E(U ) 

a finite second moment, the distribution of — - i—H approaches that of a 

normal variate with zero mean and unit variance. Hence we find the N re¬ 
quired for a test ^^ith Type I and Type II errors a and ^ by solving for N from 
the relations 

- V . 


( 1 ) 

and 


/‘Go: 

K-y/NEx{^ 




= -u» 


( 2 ) 
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Now let y = (4 J , and we find from (1) and (2) that 

\f/Moy 

N = 

fC/aVW) + UtVE,(y^) - my)\ 

1 Et{y) 

Now 


F M _ Oo) 

*■<>'> I./(.,».) 

f(x, 6t) dx 


I- 


-" ^ II * 

= AEoy^ll + o(l)] from assumption B. 


Proceeding in a similar manner, we find 

lU^a\^Eo(yy + Ufi\/Ei{yY ““ lEi(y)Y\^ = EQ('y-)[Ua + U^il + o(l))]*. 


We now have 

.aw _ iiaMi’a + 0 ( 1 ))’ .. * (hr) + <‘ - »> ‘°i! (r=i;) 

N -EMu. + v.ii + oim^ -^■|a(/) +0(1)1 

therefore 


limit 

A-*0 



l^a log 

( a ^ ( 


(f/a + UzY 
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A NOTE ON THE POISSON-CHARLIER' 

FUNCTIONS 

By C. Truesdell 
Naval Ordnance Laboratory 
The polynomials Pni'nty z) given by the definition 

(1) Pn(m, z) = (-)”* e* [e- z’^\, 

‘ This note was written while the author was employed by the Radiation Laboratory, 
M.I.T. 
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called the Poisson-Charlier polynomials, and the associated function 4'n(m, z) 
given by the definition 

(2) ^n(m, z) = z)Mm, z)y 

(3) Mm, z) s , 

m\ 

occur in statistics. Doetsch [1] has devoted a memoir to them, and they are 
noticed in Szego’s Orthogonal Polynomials (pp. 33-34). 

I suggest that they are most directly and easily studied in connection with 
the “F-equation” 

(4) -lF{ 2 ,a) = F{z,a + \), 

dz 

whose properties and application to various special functions I have sum¬ 
marized in a recent note [2]. Using the theorems of that note, which I shall 
cite by number, I shall now generalize the Poisson-Charlier polynomials and 
sketch the speediest derivation of their most interesting formal properties. 

Greek letters shall represent unrestricted real numbers, while Latin letters 
shall represent integers. 

From the existence theorem for the F-equation (Theorem 4) we know that 
there exists an integral function of z, F^{z, a), which satisfies the F-equation 
and the condition 

(5) F^(0, a) = cos(a + 0)7r • 

From the uniqueness theorem for the F-equation (Theorem 4) it follows that 

(6) Fp{z, n — ^ + J) = 0, 

(7) Ffi(z, n) = 0, n > 0. 

From the general power series solution for the F-equation (Theorem 4) we have 
the formula 

p + a + iiz)» 

We now define the Poisson-Charlier fimctions in general by the formulas 

(9) psia, z) m r(a + -a), 

(10) ^/j(a, z) a p/f(a, z). 

From the formulas (6) and (7) we see that [1, p. 263] 

(11) «)*0, n > 0; 


( 8 ) 


F/j(z, a) = cos (a + $)ir 


( 
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(12) n + /? - i 2 ) = 0, p/5(- w + /3 - i, 0 ) = 0. 

From the formula (8) we see that 

(13) pfi(a, z) = cos (/3 — a)ir 0 — a + l;z), 

whence it follows at once that 

(14) Pfiirriy z) = cos ^ir S k\( — z)'~^. 

This is the usual explicit expression for the Charlier pol 3 momials [1, p. 257]. 
From formula (13) we see that 

(15) poi-a. z) = r(l - a)z‘'y{a, z). 

In the indeterminate case when a is a negative integer we see from the formula 
(14) that 

(16) poiniy 2 ) = 1, m ^ 0. 

Hence 

(17) ^oi—dy z) = e~*y(ay 2 ), 

(18) Mm, z) = . 

From the definition (10) we now see that 

(19) ^/j(m, 2 ) = pfiimy 2 )^o(w, 2 ), 

a generalization of the formula (2). From the formula (13) and the definition 
(10) we see that 

(20) ^a03, 2 ) = cos (^ - ^ ^^ e'iP'ii-^id-p + i-yz). 

Then by Kummer’s first transformation, 

(21) ^«(/3, 2 ) = cos - a)T 1 ) ^ ^ ^ J 

from which it follows from the power series formula for solutions of the F-equa- 
tion (Theorem 4) that ^«(j5, 2 ) is a solution of the F-equation (4). 

We now have two different solutions of the F-equation based on the Poisson- 
Charlier functions: 


(A) 


F(z, a) = 6V^(-a, 2 ). 
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(B) F{z, a) = z). 

From the F-equation it is evident that 

(22) M,z) 

whence we at once deduce the formula (1). Appl 3 dng Taylor’s theorem for the 
F-equation (Theorem 8) to the solution (B) we see that [1, p. 259] 

(23) ^,(8, z + h) ; 

n.O n\ 

putting a equal to zero we find that 

(24) _si^ e—‘{ 7 (- 8 . * + « = E z), 

Zt n-o n\ 

and, more specially [1, p. 260] 

(25) 

Applying the same theorem to the solution (A) we obtain the formula 

(26) z + h) = , hiot — n, z), 

n»0 ni 

whence we recover the formula (11) by putting a equal to zero. 

Applying Theorem 9 to the solution (B) yields the result 

(27) E <Va+»C8, 2 ) = f e"Va(3, 2 + et) dd, 

n.O Jo 

which contains as a special case the formula 

(28) E <>„(m, *) = (!+ <)-”-* Q" + 1. 2(l + 

Appell’s generating expansion (see Theorem 10, part C or [3, p. 120]) applied 
to the solution (A) yields the result 


(29) 
hence 

(30) 


E Mn, 2 + y)<" = e'“ ” E Mn, y)t''; 


I :^ P 2(«.2 + 2 ,) 

fi»o n\ 


e(-^V 

ri \2 + y/ 


Pfl(», y) 


Putting y equal to zero and using the formula (13) we see that 
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(31) 2 ^ «) = 6* (l — cos I3ir, 

n.O 7l! \ z/ 

Comparing this result with the formula (25) we see that 

(32) (-)”p„(m, 2 ) = (-)’“prt(n, z). 

It would be possible to proceed in this same fashion and discover many other 
formal properties of the Poisson-Charlier functions, but it is perhaps easier to 
notice from the formula (13) that 

(33) pfiia, z) = cos (iS - a)irT(a + l) 2 ““L^-'“^( 2 ). 

LI(x) being Laguerre’s function suitably generalized for complex lower index 
[4, p. 53]. By means of this formula every relationship involving Laguerre 
functions may be translated into one involving Poisson-Charlier functions. 
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1. Random Variable!^ with Comparable Peakedness. Z. W. Birnbaxtm, 
University of Washington. 

Let U and V be random variables with symmetrical distributions, i.e. with P{U ^ —T) 
P(U ^ T) and P{V ^ -T) ^ P(V ^ T) for all T ^ 0. The random variable U shall be 
called more peaked than F if P(! | ^ T; ^ P(| F 1 ^ T) for all T ^ 0. Let Xi ,Yi and 

X 2 ,F 2 be two pairs of independent random variables such that is more peaked than Yi 
for t =» 1,2. Then under certain additional conditions X = -f X* is more peaked than 
F « Fi + n . 

2. On Optimum Tests of Composite H 3 rpotheses with One Constraint. Erich 
L. Lehmann, University of California, Berkeley. 

The problem studied is that of finding all similar and bisimilar test regions of composite 
hypotheses, and of obtaining the most powerful of these regions. Various results are ob¬ 
tained for distributions which admit sufficient statistics with respect to their parameters. 
Applications are made to the hypothesis specifying the value of the circular correlation 
coefficient in a normal population, and certain hypotheses concerning scale and location 
parameters in exponential and rectangular populations. 

3. Estimation of a Distribution Function by Confidence Limits. Frank J. 
Massey, Jr., University of California, Berkeley. 

Let Xi fX 2 f • • • , Xn be the results of n independent observations, having the same cumula¬ 
tive distribution function F(x). Form the function Sn(x) = kjn where k is the number of 
observations less than or equal to x. A confidence band Sh{x) i X/\/n will be used to 
estimate F{x). To determine the confidence coefficient it is necessary to find Pr{max V» 
1 Sn{x) — P(x) 1 ^ X/\/w}. It is sufficient to consider x uniformly distributed in the interval 
(0,1). Let X\/ n = s/t where s and i are integers. Then »Sn(x), to stay in the bandP(x) ± 
X/-n//i, can only pass through certain lattice points above x = i/tUy f == 1,2, • • • , in. The 
probability of ISn{x) passing through a particular sequence of these points is given by the 
multinomial law, and this can be summed over all permissible sequences. Limiting dis¬ 
tributions have been given by A. Kolmogoroff, and by N. Smirnoff. It is desired to test 
the hypothesis F{x) = Po(x) against alternatives F{x) *= Pi(x). Using the criterion: reject 
Fo(x) if 

max Vn 1 Po(x) — <Sn(x)| > X 

X 

the probability of first kind of error can be controlled by choice of X. A lower bound to the 
probability of second kind of error against alternatives such that max \/n 1 Po(x) — Pi(x) ] ^ 
A is given. This lower bound approaches one as n —► w. Thus the test is consistent. 

4. A Note on Sequential Confidence Sets. Charles Stein, Columbia Uni¬ 
versity, 

This paper generalizes a paper of Stein and Wald, appearing in the Annals of Math. Stat.f 
Sept., 1947. 

Let {X,}, (i « 1,2, • • Oi be a sequence of random variables whose distribution depends 
on an unknown parameter $. Sequential confidence sets are determined by senile indicating 
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when to stop sampling and a rule giving the confidence set as a function of the sample. It 
is desired that, for each sample point, the confidence set should be one of a specified class S, 
that the probability of covering the true parameter should be ^ a, and that the least upper 
bound of the expected number of observations should be minimized. If Xi are inde¬ 
pendent with the rectangular distribution on (0, 0) and S consists of all intervals of the 
form {00 , k0o) with k fixed and 0o a function of the sample, the optimum sequential pro¬ 
cedure is the classical non-sequential procedure. If the Xi are independently and identi¬ 
cally distributed in accordance with a multivariate normal distribution with known co- 
variance matrix 2 but unknown mean 0, and the confidence sets are to be of the form (d — do)' 
— do) *= r, r fixed, do a variable p-dimensional vector, a similar result holds, provided 
the desired confidence coefficient ot is not excessively small. 

6. Explicit Solution of the Problem of Fitting a Straight Line when Both 
Variables are Subject to Error for the Case of Unequal Weights. Elizabeth 
L. Scott, University of California, Berkeley. 

Let a, jS and ^i,{i = 1,2, • • • , s), be unknown fixed numbers and let 17 , =* a •+• For 
each value of i there exist m* measurements of and m measurements yik of iji , {j *» 
1,2, • • • , TWi ; A = 1, 2, • • • , nt). The variables and yik are normally distributed about 

and rii with variances ir\/ui and <rl/vi respectively, where the weights and Vi are known 
but and are unknown. The numbers w,- and n, are bounded (usually small) while s 
increases indefinitely. Thus a, /3, and <r* appear as structural parameters and the as 
incidental parameters. (See paper by J. Neyman and E. L. Scott to appear in Econometrica.) 
Modified maximum likelihood equations (MMLE) yielding consistent estimates of the 
structural parameters are tedious to solve when the products and riiVi depend on i. 
The main result of this paper consists in proving that the varying rmui and/or niVi can be 
treated as constants. Let lOi and be the harmonic means of m»Ui and niVij respectively. 
Now, MMLE’s written with niiUi *= Wi and ntVi =» Wi yield consistent estimates of a and 0. 
The asymptotic variances are also found. An application is made to certain problems of 
astronomy. 

6. Unbiased Estimates with Minimum Variance. Charles Stein, Columbia 
University. 

Let X be a random variable distributed in the space R according to one of the p.d.f.’s 
<p(x 1 0)f where 0 is an unknown parameter, and let g{0) be a real-valued function of 0. 
Let B{0) be the set of all x such that ip{x | 5) > 0 but tp {x [ 0o) =» 0, and S the set of all 0 
such that B(0) has probability 0 when 0 is the true parameter value. Let 

^(x I 0) « ^(x I 0o)Mx I 00 and A(0i ,^ 2 ) = Eiik(X I 0i) 4f{X \ 0^) | ffol 
for 01 , 0i in S. Suppose A{0\ , 0^) is everywhere finite and there exists a set function X 
of bounded variation over S such that / A{0i ^ 0^) d\{0i) ■■ g{0^. Then an estimate of 

h 

g{0)t unbiased for all in and having minimum variance at ^0 is given by f{x) » 

J ' ip{x I 0) dk{0)/ip{x I 0o). The minimum variance is / g{0) d\{0) — [^(®o)l*. If the 
a * Js 

definition of f{x) is modified at a set having probability 0 when 0 — 0o, the properties on S 
and at 0o remain unchanged. Under mild restrictions this alteration can be carried out so 
as to make f(x) an unbiased estimate of 0 for all S, The results are related to the work of 
Fisher, Dugu 6 , Rao, and Bhattacharyya on the amount of information. 

7. Sufficient Statistics and a System of Partial Differential Equations. (A 

Contribution to the Neyman-Pearson Theory of Testing Hypotheses.) Pre- 
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liminary Report. Erich L. Lehmann, University of California, Berkeley, and 
Henry Schepf^, University of California, Los Angeles. 

In the Ne 3 rman-Pear 8 on theory of testing hypotheses the problem of the existence and 
determination of similar regions has been treated under two approaches: (1) Assuming the 
existence of a set of sufficient statistics for the nuisance parameters; (2) Assuming that the 
probability density satisfies a certain system of partial differential equations. By solving 
the differential equations it is now shown that they imply the existence of sufficient statistics 
for the nuisance parameters. Knowledge of the form of the solution of the differential 
equations permits simplification of the known theory of optimum tests (type B, Bi , etc.) 
as well as some generalization. 


8. Power Function of the Analysis of Variance and Covariance Test of a 
Normal Bivariate Population. W. M. Chen, University of California, Berkeley. 

The problem of finding the power function of the analysis of variance and covariance test 
of a normal bivariate population, p « 0 and « o-i, by means of principle of likelihood 
was reduced to the determination of the distribution function P{L) of the following moment 
problem: 


f 

Jo 


V - dPiL ) 


(1 - 




^ik,T y 


(k - 1 , 2 , ...), 


where 


4.r(. - I +,) rr+,) 
r+ r^r(n - 1 + 2* + r) 

and a, the argument of the power function, lies in the interval (0,1) and vanishes only when 
the hypothesis tested is true. The moment problem was found and solved by rather tricky 
methods. The result is 


PiL) 




,5 + 1 


) 


where b =» 

9. A Mathematical Model of the Relation between White and Yolk Weights 
of Birds’ Eggs. G. A. Baker, University of California, Davis. 

The purpose of such a model is to find a rational method of estimating a *^best line** in 
soma sense which will represent the relation between white and yolk weights for some or all 
species of birds. From data at hand it appears that birds within species may differ in 
means and variances of weights and that the yolk and white weights are positively corre¬ 
lated . Yolk and white weights within a species are functions of egg number. The standard 
deviations of yolk and white weights for different species are approximately proportional 
to mean values. The “true** means for yolk and white weights for different species do 
not lie on a line because of biological differences between species with the same egg size. 
The standard deviation of species deviations from a straight line depend on the size of the 
egg (may be proportional to a weighted sum of the yolk and white weights). If sampling 
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variances are sufficiently small they may be neglected and a straight line fitted assuming 
both variables subject to error and non-uniform variance. The practicality of maximum 
likelihood estimates is considered. 

10. Statistical Analysis for a New Procedure in Sensitivity Experiments. A. 

M. Mood, Iowa State College, and W. J. Dixon, University of Oregon. 

In the language of biological assay the sensitivity experiment investigates the proportion 
of subjects that respond to a given concentration, x, of a certain chemical. It is assumed 
that only one test may be made on each subject. The new procedure is characterized by a 
change in x for each successive test, depending on the result of the preceding test, x is 
reduced to the next lower of a fixed set of concentrations for the next test if there is no 
response and is increased to the next higher concentration if there is a response. Observa¬ 
tions are thus concentrated near the mean and few tests are made for values of x where a 
very large or very small proportion of subjects would respond. Assuming x is normally 
distributed, approximate maximum likelihood estimates are obtained for the mean and 
standard deviation of x. These assume a form which is simple to compute. Choice of op¬ 
timum increments of x for various situations is investigated. 

11. The Relation of Inbreeding to Calf Mortality. W. M. Regan, S. W. 
Mead, and P. W. Gregory, University of California, Davis. 

An analysis of calf mortality in the University of California dairy cattle breeding experi¬ 
ment is presented. Calves up to 4 months of age that were born singly are included in the 
study. Only those stillbirths and abortions from cows free from Brucellosis and health 
and reproductive abnormalities were considered. A total of 774 Jersey and 258 Holstein 
calves were included. Calves were classified according to inbreeding coefiicients as follows 
Class I, the controls 0 0 to 0.1249; Class II, 0.125 to 0.2448; Class III, 0.245 to 0.3749; and 
Class IV, 0.375 and over. There was no relation between the number of abortions and the 
degree of inbreeding. The stillbirths, too few to be statistically significant, tend to increase 
as the coefficient of inbreeding increased. Following birth, however, mortality was corre¬ 
lated with inbreeding of both males and females but for the males it was greater than for 
the females in Classes III and IV, but the difference is hardly significant. The Jerseys 
tended to be less viable than the Ilolsteins. Some of the increased mortality of the more 
highly inbred animals could be accounted for by the action of two lethal genes; one con¬ 
trolling an anomaly of the liver, the other an anomaly of the heart; there was no plausible 
explanation for most of it. Within sex, inbreeding class, and breed there w^as considerable 
variation in the mortality of the progeny of different sires. Some of these differences were 
statistically significant. 

12. Observations on Designs for Cooperative Field Tests. P. A. Minges, 
University of California, Davis. 

In California conditions vary so greatly between the principal production areas that it 
is necessary to establish experimental plots in each of the areas if reliable information is 
to be obtained regarding cultural practices. Most of these tests must be conducted on 
ranches in cooperation with growers and local agricultural extension agents. The designs 
of these tests should be relatively simple, the arrangement should be adjustable to work 
into the growers’ cultural practices and to permit the obtaining of yield records with a 
minimum of interference to the growers’ operations, yet the design must be adequate to 
yield valid data. The randomized block design has proved the most useful, although paired 
plots, factorials, split-plots and Latin squares have been used successfully under certain 
conditions. The Iiatin square design is useful when a two-way variation is expected, other- 
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wise it is not usually very efficient. Where yield data are of prime importance, for ureplica- 
tions have been considered most practical. In tests such as variety trials when factors 
other than yields are important, two replications may be adequate. The size of the plot 
has been varied to fit the crop, conditions of the field, and known soil variability. Plots 
two rows wide and 50 to 135 feet long often have been used, frequently without guards 
between plots. Since it is desirable to include checks (untreated controls) in most tests, 
small plots will reduce the loss to the growers when the treatments prove beneficial. The 
information derived from these tests is of most interest to growers and county agents so 
the data should be presented in tables that are easily read. The variability figure which is 
confusing to most people probably can best be presented as the least significant difference. 

13. Population Genetics. N. H. Horowitz, California Institute of Tech¬ 
nology. 

Population genetics attempts to describe the effects on the genetical structure of Mende- 
lian populations of factors such as mutation, selection, migration, and random fluctuations 
due to sampling errors. These diverse elements are brought under a common viewpoint by 
considering their effects on gene frequencies. Since change in gene frequency is the ele¬ 
mentary process of evolution, the above factors are causal agents of evolution. Mathe¬ 
matical models illustrating the interplay of the various elements have been constructed by 
Wright, Haldane, and Fisher. The nature of Mendelian inheritance is such that gene 
frequencies remain constant in large populations not subject to net mutation, selection, or 
migration pressures. Unbalanced pressures initiate evolutionary changes which continue 
until equilibrium is reached at a new level of gene frequencies. Equilibrium frequencies are 
determined by opposing pressures—e.g., opposing mutation rates, mutation opposed by 
selection, etc. Equilibrium, stable or unstable, is also possible under selection alone. In 
small populations, sampling errors among the gametes produce random fluctuations in gene 
frequencies which, superimposed on the equilibrium values, result in probable distributions 
of frequencies. The latter provide a mechanism for the evolution of characters, especially 
biochemical syntheses, which depend on the simultaneous action of a number of individually 
non-adaptive genes. 


14. The Choice of Inspection Stringency in Acceptance Sampling by Attributes. 

J. L. Hodges, Jr., Uiiiversity of California, Berkeley. 

In acceptance sampling by attributes, the probability p that an item will be defective is 
taken to be a function g{Zf y) of the quality x of the population and the stringency y of 
inspection. Let n, the number of items inspected, be fixed, and reject if the number of 
defectives is ^ A;. It may then be possible to satisfy a condition on the power function 
with different values of A:, by adjusting y properly. This paper is concerned with the choice 
of k and y in such situations. A criterion is given, and it is shown that the criterion is 
approximately satisfied by A; « [ng(xQ , y)] where Xq separates acceptable and non-acceptable 
values of x, and y maximizes 


«ff(xo, y) / -rr- 

j V v)[l 


ff(xo, y)l- 


An asymptotic property of this approximation is shown. The method is applied to two 
examples: (a) testing the mean bacterial density of a liquid by the dilution method, y 
being the volume of liquid incubated, and (b) testing the variance 2 of a normally dis¬ 
tributed dimension of known mean m by applying gauges set at m db * . The approximate 

y 

solution is found to be satisfactory in both cases for m » 20. 
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15. The Application of Learning Curves to Industrial Planning. Preliminary 
Report. James R. Crawford, Lockheed Aircraft Corporation. 

Learning curves are significant factors of analysis in industries producing quantities of 
less than 20,000 units of a given article. Ship-building and airframe manufacture are the 
two largest industries in this class. Learning curves occur where job costs are kept either 
by individual unit or by lot, and also where achievement is measured against a standard. 
Cost per unit plots against ordinal unit number as a straight line on logarithmic graph- 
paper. Learning curves are used to supplement time-studies, determine the capacity of 
tooling, layout of budgets, and for estimating and bidding. The experience of individual 
workers and management are reflected in these analyses. The slope of the learning curve 
is related to the amount to be learned. Plateaus occur which are related to the hiring of 
new’ w’orkers and to the relaxing of control measures. Other consistent minor patterns 
occur which are related to specific conditions. Equations have been derived and tables 
computed for five related forms of the learning curve. Graphic methods are satisfactory 
except for bidding. This study covers a simple approach to an important problem of indus¬ 
trial management. The findings in the industrial field may benefit research in the field 
of the psychology of learning. 

16. Relative Efifects of Inbreeding and Selection in Poultry. W. 0. Wilson, 
University of California, Davis. 

Egg production rate, fertility, hatchability, and chick mortality records from the Iowa 
State College Poultry Department’s inbreeding project were studied. Statistics which 
were calculated from the data included simple and partial regression of traits on inbreeding, 
estimates of heritability by correlation between paternal half-sibs and by daughter-dam 
regressions, and selection differentials. The net genetic gain or loss in merit per generation 
w’as considered to be the sum of the product of selection differentials and heritability, plus 
the product of regression of trait on inbreeding and increase in amount of inbreeding. The 
amount of inbreeding that can be done in each of the traits was estimated when there was no 
net loss or gain. Of the traits studied, the rank w’as in the following order: Hatchability, 
chick mortality, fertility, and egg production. 

17. The Rate of Genetic Gain in Egg Production in Progeny-Tested Flocks 
as a Fimction of the Interval between Generations. Everett R. Dempster and 
I. Michael Lerner, University of California, Berkeley. 

The rate of genetic gain in a character for which selection is practiced depends in addition 
to the intensity of selection on (1) the accuracy of selection, and (2) the average interval 
between generations. These factors are not independent and exercise a pull in opposite 
directions. Through the application of Wright’s technique of path coefficients comparisons 
can be made between the expected rates of genetic gain in populations containing varying 
proportions of breeding animals of different ages. The methods used involve the estimation 
of correlations between genotypes, and various selection indexes based on individual, sib 
and progeny records in inculled populations as well as in populations whose range has been 
restricted by previous selection. From these estimates the relative efficiencies of different 
age distribution schemes of a breeding population can be determined. A specific solution 
for such a situation in a flock bred for egg production will be presented as an illustration of 
the problems and methods used in the study of the genetics of populations under artificial 
selection. 

18. Statistical Criteria of tfie Effectiveness of Selective Procedures. Prelim¬ 
inary Report. R. F. Jabrbtt, University of California, Berkeley. 
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The “validity coefficient,” the standard error of estimate, the index of predictive effi¬ 
ciency, the “selection ratio” of Taylor and Russell, Johnson’s Gamma, and other statistical 
devices have been suggested as indices of the effectiveness of selective programs. These 
devices all suffer from the deficiency that they do not permit a satisfactorily precise estimate 
of the dollar value of the increased output expected from the selection program and thus 
leave unsettled the question as to whether or not the cost of such a program is justified. 
The relationship between the correlation coefficient on the one hand and the mean value of 
y for an unselected population (F being an objective output-type criterion), the standard 
deviation of V for an unselected population, and the mean value of V for the upper Np in¬ 
dividuals selected on the basis of their high performance on the selective test X on the other 
hand, provides the basis for estimating the increase in the mean output of a group of workers 
selected on the basis of a testing program yielding any specified validity coefficient with the 
criterion Y. Increase in productivity of selected workers is shown to be a function of the 
validity coefficient, the rigorousness of selection, and the coefficient of variability of the 
output criterion among “unselected” employees. 

19. Approaches to Univocal Factor Scores. Preliminary Report. J. P. 
Guilford, University of Southern California. 

In spite of the fact that uni vocal factor scores are badly needed for various reasons, it 
appears to be impossible by present methods to construct pure tests for some common 
factors. Recourse must therefore be made to statistical control of component variances. 
It is desirable to derive each factor score from a minimum number of tests. The availability 
of a few uni vocal tests makes this requirement fairly easy to satisfy. Such tests serve well 
as suppression variables for their common-factor variances where not wanted in other tests. 
Several principles may be invoked as objectives: (1) to maximize the desired variance in 
the impure test, (2) to reduce the undesired variance to zero, or (3) to minimize the undesired 
variance without intolerable loss of the desired variance. A secondary objective is to 
assure a combining weight of +1.00 for the test measuring the desired factor. Equations 
for achieving the objectives have been derived and the limitations and implications of each 
procedure have been noted. By means of statistical control, the situation seems hopeful 
for the achievement of univocal scores for a fairly large number of unique psychological 
variables. There are implications for experimental psychology as well as for vocational 
testing. 

20. A Note on the Problem of Binary Stars. Elizabeth L. Scott, Uni¬ 
versity of California, Berkeley, 

This paper concerns some of the problems of Trumpler (see next abstract), is the 
radial velocity of the i-th star, t«l, 2,* • •, a, at selected at random, j = 1, 2,* • -, 71 . Xij, 
measurement of is N (f,y, <r,). is random with distribution c “• ^*o)*) 

where ki ^ 0 and are unknown. (1) Test of hypothesis that ki « O. Case (i) o-,- known. 

Whatever the exact test T, its power firik) has derivative /3r(0) ■■ 0. Test maximizing 

n 

/3r(0) is that of Trumpler with criterion (j;</ -- x< )* > . Case (ii) un- 

j-i 

known. Whatever the exact test t, ■■ 0, m »■ 1, 2, 3. Test maximizing jSj^Uo) is 

n r n "Ij n 

Trumpler’s test (xi,- — Xi-)^ > (2) Let ^ ““ ^*o)* = 2X<«r*. 

y-i Lj-1 j /-I 

For constant velocity stars X — 0. For others it is a random variable. Since, given X >■ 0, 

S* is distributed as non-central x*, an integral equation connects the distributions of <S* 
and X. Its solution yields an estimate of the proportion of constant velocity stars. After 
estimating the distribution of X, the level of significance can be estimated and also the 
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number n of measurements so that the proportion of constant velocity stars declared vari¬ 
able will be less than p, specified in advance. 

21. Statistical Problems of Spectroscopic Binaries. Robert J. Trumpler, 
University of California, Berkeley. 

Spectroscopic Binaries are stars whose radial velocities, as measured by the Doppler 
shift of spectral lines, show a periodic variation. The first problem is to obtain a statistical 
criterion for deciding whether a star with several radial velocity measures, made at different 
times, has a high probability (larger than a specified limit) of variable velocity and should 
be announced as an object worthy of further study. The second problem is to find the 
percentage of variable velocity stars among a large list of stars with several radial velocity 
measures for each star. From the distribution of standard errors only the percentage of 
cases where the velocity variation exceeds a certain limit can be ascertained. The third 
problem is concerned with those stars for which a binary orbit has been determined. The 
statistical distribution of these binary systems according to mean distance between the two 
stars and the ratio of their masses can be evaluated within certain limits. 



NEWS AND NOTICES 


Readers are invited to submit to the Secretary of the Institute news items of interest 

Personal Items 

Mr. Kenneth J. Arrow has been appointed Research Associate of the Cowles 
Commission. 

Dr. W. D. Baten, formerly of Michigan State College, is now Chief, Opera¬ 
tions Branch, Planning Section, Air Defense Command, Mitchel Field, New 
York. 

Dr. Paul T. Bruyere is now Chief of the Medical Records and Statistics Branchy 
Army Institute of Pathology, Office of the Surgeon General, War Department. 

Dr. A. C. Cohen received his discharge from the Army, with the rank of lieu¬ 
tenant Colonel, at the beginning of the spring quarter, and returned to his former 
position at Michigan State College. He has accepted a position at the Uni¬ 
versity of Georgia beginning with the 1947 summer session there. 

Dr. Hallctt H. Germond has resigned from his position as professor of mathe¬ 
matics at the University of Florida. He is now Director of Research for the 
S. W. Marshall firm of Consulting Engineers, in New York City. 

Dr. Meyer A. Girshick, formerly with the Department of Agriculture, is now 
with the Douglas Aircraft Company in Santa Monica, California. 

Dr. Clyde H. Graves has accepted a position as Operations Analyst, Opera¬ 
tions Analysis, Air Defense Command, Mitchell Field, New York. 

Dr. E. J. Gumbel has been appointed to an Associate Professorship at Brook¬ 
lyn College. 

Dr. Trygve Haavelmo has returned to Norway, and is at the University 
Institute of Economics, Oslo. 

Mr. Joseph O. Harrison, Jr., is now employed as a Mathematician in the 
Computing Branch of the Ballistic Research Laboratories, Aberdeen Proving 
Ground. 

Dr. Wassily Hoeffding has accepted a psoition as Research Associate, The 
Institute of Statistics, University of North Carolina, Chapel Hill. 

Mr. Cyrus A. Martin is now an administrative analyst and statistician, as¬ 
sisting Chief of Personnel Control of Signal Corps, in Washington, D. C. 

Mr. Jack I. Northam has accepted an Assistant Professorship in the Depart¬ 
ment of Mathematics, Kansas State College, Manhattan, beginning with the 
1947 summer session. 

Professor Henry Scheff^, who has been on leave for the past year, returned to 
his position in the Engineering Department, University of California at Los 
Angeles, in June. 

Mr. Edward M. Schrock has accepted a position as Quality Control Engineer 
with the General Electric Company at their Erie Works, Erie, Pa. 

Mr. Jerome R. Steen, who has been manager of Quality Control Engineering 
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with the Sylvania Electric Products in Emporium, Pa,, has now transferred with 
the same company to Flushing, New York. 


Professor Emeritus Irving Fisher, of Yale University, died April 29, 1947, 
at the age of eighty. 


In connection with the Atlantic City meeting of the American Chemical 
Society, Apnl 14-18, 1947, a symposium on Statistical Methods in Experimental 
and Industrial Chemistry was held, in which several members of the Institute 
of Mathematical Statistics took part. The following program was presented 
Tuesday morning and afternoon, April 15: 

(1) Introductory Remarks. B. L. Clarke. 

(2) The Management Viewpoint George Smith. 

(3) A New Technique for Testing the Accuracy of Analytical Data. W. J. 

Youden. 

Discus&-ion: Grant Wernimont, R. F. Moran, John Mandel, and Roland 
H. Noel 

(4) Design of Experiments in Industrial Research. Hugh M. Smallwood. 

(5) Statistical Training for Industry. Samuel S. Wilks. 

Discussion: John Tukey, E. V. Lewis, Churchill Eisenhart, and C. West 
Churchman. 


Preliminary Actuarial Examinations 
Prize Awards 

The winners of the prize awards offered by the Actuanal Society of America 
and the American Institute of Actuaries to the nine undergraduates ranking 
highest in the combined score on Part 1 and Part 2 of the 1947 Preliminary 
Actuarial Examinations are as follo^\s: 


Fini Prize of S200 
James H. Chung 
Additional Prizes of 9100 

James F. A Biggs 
George Y. Cherlin 
Frank H David 
Thomas M Galt 
Charles F Pinzka 
Philip C. Rapp 
Morton K Schwartz 
James G C. Templeton 

The two actuarial organizations 
awards for the 1948 Examinations. 


University of Toronto 

Yale University 
Rutgers University 
Harvard University 
University of Manitoba 
Rutgers University 
University of Buffalo 
Brown University 
University of Toronto 

authorized a similar set of nine prize 
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The Preliminary Actuarial Examinations consist of the following three examina¬ 
tions : 

Part 1. Language Altitude Examination 

(Reading comprehension, meaning of words and word relationships, antonyms, 
and verbal reasoning). 

Part 2. General Mathematics Examination 

(Algebra, trigonometry, coordinate geometry, dilferential and integral 
calculus). 

Part 3. Special Mathematics Examination 

(Finite differences, probability and statistics). 

The 1948 Examinations will be administered by the College Entrance Examina¬ 
tion Board at centers throughout the United States and Canada on May 14-15, 
1948. 


Correction 

In the Directory of Members published in Vol. XVII, No. 4 (December 1946) 
Professor Joseph Kamp4 de FerieUs name is listed in the F’s under Feriet. It 
should have appeared in the K's, under Kamp6 de Feriet. 


New Members 

The following persons have been elected to membership in the Institute (March 1 to May 30, 

1947): 

Adams, Joe K. Ph. M. (Wisconsin) Graduate student and half-time instructor in Psy¬ 
chology, Graduate College, Princeton University, Princeton, N. J. 

Adams, Walter B. Communications Analyst, Civil Aeronuatics Admin., Dept, of Com¬ 
merce, 825S S. Ingleside Ave.j Chicago 19^ III. 

Aitken, Alexander C. D.Sc. (Edinburgh) Professor of Mathematics, University of Edin¬ 
burgh, 23 Stirling Roadf Edinburgh 5, Scotland 

Brambilla, Francesco Ph.D. (Univ. L. Bocconi) Lecturer in Math. Statistics, Institute 
of Statistics, University L. Bocconi, 6 via Panzacchi, Milano, Italy 

Brown, George Middleton, D.Sc. (Michigan) Asst. Prof, of Math., Mich. State College, East 
I.Ansing, Mich., 633 Cherry Lane 

Bueno, Luiz de Freitas, E.E. (Mackenzie Coll.) Professor da Universidade de Sao Paulo, 
Brazil, Rua Itambt 341, Casa 13 

Burke, Cletus J., M.A, (U.C.L.A.) Res. AssH, Univ. of Iowa, Iowa City, Iowa, 118 River¬ 
side Park 

Cameron, Joseph M., M.S. (N. Car. State) Room 302 South Building, National Bureau of 
Standards, Washington, D. C. 

Carpenter, Osmer M.S. (Iowa State) Instructor, Mathematics Department, Iowa State 
College, Ames, Iowa 

Castellani, Maria D.Sc. (Rome) Visiting Professor, Department of Mathematics, Uni¬ 
versity of Kansas City, Kansas City 4, Mo 

Chemoff, Herman Sc.M. (Brown) National Research Council Pre-Doctoral Fellow, 3003 
Wallace Ave., Bronx 67, N. Y. 

Clark, Stanley M.Ed. (Saskatchewan) Student and teaching assistant, 1301-7ih St., S.E., 
Minneapolis 14, Minn. 
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Cover, John H. Ph.D. (Columbia) Director, Bureau of Business and Economic Res., 
Univ. of Maryland, College Park, Md. 

Dailey, John T. M.S. (N. Texas Teachers Coll.) Res. Psychologist (Aviation), Psycho¬ 
logical Res. and Examining Unit, Sqn. E, Indoctrination Div., Air Training Command, 
San Antonio, Texas 

Darling, Donald A. Ph.D. (Calif. Inst. Tech.) Teaching Ass't, Calif. Inst, of Technology, 
Pasadena 4, Calif. (As of July 1947, Dept, of Math., Cornell Univ., Ithaca, N. Y.) 

Darmois, Georges D.Sc. (Paris) Prof. h. la Faculty des Sciences de Paris, 7 Rue de VOd^oUf 
Paris 6 y France 

Davies, J. Alfred M.A. (Alabama) Statistician, Design Eng. Section, General Electric 
Co., 708 Hill Avenuey OwneshorOy Kentucky 

Dunnett, Charles W. M.A. (Toronto) Student, 1044 John Jay Hall, Columbia Univ., 
New York 27, N. Y. 

Egermayer, Frantisek Sc.D. (Charles Univ., Prague) Chief of Section, State Statistical 
Office, 2 BilskihOy Prague VI ly Czechoslovakia. 

Fickenscher, Edgar H. A.B. (Calif.) Graduate student and teaching ass’t, Univ. of Calif., 
H90 Acton St.y Berkeley 2y Calif. 

Fraga, Constantino G. Jr. (Sao Paulo) Head, Dept, of Statistics, Institute Agronomico, 
Campinas (S.P.), Brazil. 

Frank, Elmore J. B.A, (Chicago) Instr. in Statistics, Ill. Institute of Tech., and Statisti¬ 
cian, Commercial Res. Dept., Armour and Co., 6Jl^2S Maryland Ave.y Chicago 15 y III. 

Frisch, Ragnar Ph.D. (Oslo) Professor, Universit}’' Institute of Economics, Oslo, Norway. 

Geary, Robert C. D.Sc. Superintending Officer, Statistics Branch, Dept, of Industry and 
and Oimmcrco, 27 I^eeson Park, Dublin, Ireland. 

Goodman, John R. M.S. (Iowa State) Head, Sampling Section, Survey Res. Center, 
Univ. of Mich., .\nn Arbor, Mich. 

Gutman, Pierre M.A. (Columbia) Student, 7 Mountain Ave.y Maplewood y N. J. 

Hartline, H. K, M.D. (Johns Hopkins) Assoc. Prof, of Biophysics, Johnson Res. Founda¬ 
tion, Univ. of Pennsylvania, 36th and Spruce Sts., Philadelphia, Pa. 

Hartog, Jacob A. (Rotterdam) Rockefeller Fellow, 25 Pollen St.y CambridgCy Mass. 

Jacobs, Marcus A.B. (Penn.) Health Stafisticisn, 44^9 S. 36lh St.y Arlington, Va. 

Jeeves, Terry A. A.B. (Calif.) Teaching ass’t in math., Univ. of Calif., 2511 Hearst Ave.y 
Berkeley 9, Calif. 

Kempthome, Oscar M.A. (Cambridge, England) Res. Assoc. Prof., Statistical I^ab., Iowa 
State College, Ames, Iowa 

Kendall, David G. M.A. (Oxford) Fellow, Magdalen Coll., Oxford, England 

Kent, Leonard M.B.A. (Chicago) Instr. in Statistics, School of Business, Univ. of Chic¬ 
ago, Chicago 37, Ill. 

Kupperman, Morton B.S. (C.C.N.Y.) Statistician, Office of the Surgeon General, War 
Dept., 2829-27th St., N.W., Wa.Hhington 8, D. C. 

Lhati, Elizabeth L. M.A. (Michigan) Statistician, Bur. of Measurement and Guidance, 
Carnegie Institute of Technology, Pittsburgh 13, Pa. 

Levine, Harry D. B.S. (Chicago) Instr., I^ong Island Univ., 164 W. 96 St., New York 25, 
N. Y. 

Lichtenstein, Morris B.A. (Michigan) Statistician, 4811 N. Capitol St., N.E., Washington 
11, D.C. 

McMillan, Brockway Ph.D. (Mass. Inst. Tech.) Member, Technical Staff, Bell Tele¬ 
phone Labs., Murray Hill, N. J. 

Marshall, Andrew W. Student, 5757 University Ave., Chicago 87, III. 

Metzner, Charles A. Ph.D. (Wisconsin) Study Director, Survey Research Center, Univ. 
of Michigan, Ann Arbor, Mich. 

Norton, John W. B.S. (California) Lab. Supervisor, Union Oil Co. of Calif., 5529 Mac¬ 
donald Ave.y Richmond, Calif. 
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Otter, Richard Ph.D. (Indiana) Instructor, Fine Hall, Princeton Univ., Princeton, N. J. 

Passos, Helena Rocha Penteado Dirctor de Divisao do Depto. Estadual de Estatistica de 
Sao Paulo, Avenida Angelica^ 160y Apto. d, Sdo Paulo, Brazil 

Priest, Edward I. B.S. (Columbia) Student in mathematics, 1204 E. 55lh St., Chicago, 111. 

Quensel, Carl-Erik Fil.Dr. (Lund) Prof, at the University, Lund, Sweden, Linnegaian 
14 

Rankin, Mozelle M.A. (Ohio State) Ass’t Instructor, Ohio State Univ., 107-14lh Ave., 
Columbus 1, Ohio 

Robb, Richard A. D.Sc. (Glasgow) Mathematics Lecturer and Mitchell I^ecturer in 
Statistics, Univ. of Glasglow, Glasgow, W. 2, Scotland. 

Ruist, Erik Fil.kand. (Stockholm) Amanuens, Industriens utredningsinstitut, Stock¬ 
holm 16, Sweden 

Shani, Inder M. M.A. (Punjab) Rothamsted Experimental Station, Harpenden, Herts, 
England. 

Schneider, B. Aubrey Sc.D. (Johns Hopkins) Ass’t Director, Dept, of Statistics and 
Special Services, American Cancer Society, 47 Beaver St., New York 4, N. Y. 

Seitz, Jiri Ph.D. (Prague) Koufiniskd 8, CSll, Praha XII, Czechoslovankia. 

Simaika, Jacques B. Ph.D. (Ijondon) Lecturer, Faculty of Science, Fuad I University, 
Abbassia, Cairo, Egypt. 

Slatin, Benjamin M.A. (Columbia) Jr. Analyst, Econometric Institute, 179 Peshine 
Avc., Newark 8, N.J. 

Suydam, Bergen R. A.B. (N.Y. State Coll, for Teachers) Graduate student, Columbia 
University, 1 W. 706 St., Shanks Village, Orangeburg, N. Y. 

Tashmuhamed, Sarymsakov Ph.D. (Moscow) President of the Academy of Sciences of 
Uzb.SSR, Professor of the University, Tashkend, ul. Abdulli Tukaeva I, Tashkent, 
USSR 

Travers, Robert M. W. Ph.D. (Columbia) Examiner, and Assoc. Prof, of Education, 
Bureau of Psychological Services, Univ. of Mich., Ann Arbor, Mich. 

Weiner, Sidney B.S. (C.C.N.Y.) Student, New York University Graduate School, 15S9 
East 17th St., Brooklyn SO, N. Y. 

Wezelman, Sol M. A.B. (Michigan) Graduate student. University of Michigan, Ann 
Arbor, Mich., 2432 Burt St., Omaha, Nebr. 

Wishart, John D.Sc. (London) Reader in Statistics, School of Agriculture, Cambridge, 
England 



REPORT ON THE NEW YORK MEETING OF THE INSTITUTE 

The Twenty-Sixth Meeting of the Institute of Mathematical Statistics was 
held in New York City on Thursday, April 24, and Friday, April 25, 1947, and 
was co-sponsored by the American Mathematical Society. This meeting was 
devoted to a program on Stochastic Processes and Noise. The attendance of 
190 persons included the following 75 members of the Institute: 

F. A. Acton, C. B. Allendoerfer, F, L. Alt, T. W. Anderson, Jr., L. A. Aroian, W. D. Baten> 
Robert Bechhofer, J. H. Bigelow, D. H. Blackwell, Paul Boschan, G. W. Brown, R. S. Bur- 
ington, B. H. Camp, E. W. Cannon, A. G. Carlton, K. L. Chung, P. C. Clifford, D. D. Cody, 
Harald Cramer, II. B. Curry, J. H. Curtiss, R. L. Dietzold, J. L. Doob, Jacques Dutka, 
Churchill Eisenhart, Benjamin Epstein, Will Feller, M. M. Flood, Bernard Friedman, C. P. 
Gerschenson, H. II. Goode, C. H. Graves, E. J. Gumbel, T. E. Harris, Millard Ilastay, L. H. 
Herbach, P. G. Hoel, Mark Kac, R. D. Keeney, T. C. Koopmans, William Kruskal, Jack 
I^aderman, J. E. Lieberman, S. B. Littauer, Melitta Lowy, P. J. McCarthy, Brockway Mc¬ 
Millan, Frederick Mosteller, L. F. Nanni, P. M. Neurath, G. E. Noether, M. L. Norden, C. 
O. Oakley, P. S. Olmstoad, G. B. Price, J. S. Rhodes, John Riordan, Selby Robinson, Frank 
Saidel, Arthur Sard, F. E. Satterthwaite, G. R. Seth, C. E. Shannon, Jack Sherman, W. A. 
Shewhart, Rosedith Sitgreaves, Andrew Sobezyk, Milton Sol^l, Emma Spaney, C. M. 
Stein, J. W. Tukey, D. F. Votaw, Jr., B. T. Weber, S. S. Wilks, Jacob Wolfowitz. 

The first session, was held on Thursday morning, with Professor Carl Al¬ 
lendoerfer of ITaverford College serving as chairman. The following program 
was presented: 

Stochastic Processes — 

Description, Professor J. L. Doob, Columbia University 

Estimation, Professor Will Feller, Cornell University 

Prediction, Professor N, Wiener, Massachusetts Institute of Technology 

This meeting was concluded with a discussion by Dr. H. W. Bode, Bell Telephone 
Laboratories, Professor Mark Kac, Cornell University, and Professor A. Wald, 
Columbia University. 

Dr. S. O. Rice, Bell Telephone Laboratories, was chairman of the Thursday 
afternoon session. The following program was presented: 

Stochastic Processes in Some Applications — 

In Economics, Dr. T. Koopmans, Cowles Commission 
In Insurance, Professor H. Cramdr, Yale University 
In Cosmic Radiation, Professor N. Arley, Princeton University 
In Nuclear Physics, Dr. S. M. Ulam, Ixjs Alamos Laboratory 

The final session was held on Friday morning with Professor J. W. Tukey 
of Princeton University as chairman. The program was as follows: 

Different Ways of Describing Noise — 

By a Noise Spectrum, Dr. C. E. Shannon, Bell Telephone Laboratories 
By a Single Function, Mr. J. E. Bigelow, Institute for Advanced Study 
By Many Functions, Professor Mark Kac, Cornell University 
Round Table on Interrelations, Messrs. Shannon, Bieglow, Kac, and Rice 

P. S. DWYER, 
Secretary. 
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REPORT ON THE APRIL MEETING OF THE INSTITUTE IN 
ATLANTIC CITY 

The Twenty-Seventh Meeting of the Institute of Mathematical Statistics was 
held in cooperation with the Eastern Psychological Association, on Saturday 
morning, April 26, 1947, in Atlantic City. This meeting was a Round Table 
on Certain Recent Statistical DevelopmentSy and its attendance of approximately 
100 persons included the following 9 members of the Institute: 

F. S. Acton, J. W. Dunlap, Benjamin Epstein, Irving Ijorge, P. J. McCarthy, Frederick 
Mosteller, P. J. Rulon, F. E. Satterthwaite, and Emma Spaney. 

Professor Bernard Riess of Hunter College was chairman of the meeting. 
The following program was presented: 

Papers: Sequential Analysis. 

Dr. Irving Lorge, Teachers College, Columbia University 
Staircase Methods. 

Dr. Philip J. McCarthy, Cornell University 
Inefficient Statistics. 

Dr. Frederick Mosteller, Harvard University 

Discussion: Dr. Jack W. Dunlap, Psychological Corporation 

Dr. Leon Festinger, Massachusetts Institute of Technology 

Dr. William E. Kappauf, Princeton University 

Dr. Joseph Zubin, New York Psychiatric Institute Hospital 

P. S. DWYER, 
Secretary. 
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REPORT ON THE SAN DIEGO MEETING OF THE INSTITUTE 

The first Western Rep^onal meeting of the Institute of Mathematical Statistics 
was held in San I3iego, California, June 17-19, 1947, jointly with the American 
Association for the Advancement of Science. The meeting ^^as attended by 53 
persons, including the following 31 members of the Institute: 

G. A. Baker, Joseph Berkson, Z. W. Birnbaum, H. C. Carver, Harald Cram4r, J. R. 
Crawford, Dorothy Cruden, W. J. Dixon, Robert Dorfman, M. W. Eudey, IGvelyn Fix, 
John Gurland, J. L. Hodges, Jr., J. M. Howell, H. M. Hughes, E. S. Keeping, E. L. Lehmann, 
R. H. Lien, F. J. Massey, G. F. McEwen, Frederick Mosteller, S. W. Nash, Jerzy Neyman, 
Kathryn B. Rolfe, Henry Scheff^, Herbert Solomon, C. M. Stein, Zenon Szatrowski, H. M. 
Walker, J. D. Williams, Zivia S. Wurtele. 

The afternoon session on June 17 was a joint meeting ^vith the Group of 
Former Operations Analysts. The follo\\ing program was presented under the 
chairmanship of Col. Roscoe C. Wilson: 

Topic: Statistical Problems in Operations Analysis. 

Papers: Engineering and Statistics at the Pacific Front in World War II. 

Roger Wilkinson, Bell Telephone Laboratories, New York City. 

Present Organization and Activities of Operations Analysis. 

Leroy A. Brothers, Operations Analysis, Asst. Chief of Air Staff-3, Washington, 

D. C. 

Statistical Evidence of Bomb Release Malfunctions. 

Mark W. Eudey, University of California, Berkeley. 

Study of Effectiveness of Certain Bombs Used Against German Industrial Targets. 

J. Neyman, University of California, Berkeley. 

The morning session on June 18 was presented with Professor Alva R. Davis 
as chairman, and the program was as follows: 

Topic: Statistical Problems in Biology. 

Papers: A Mathematical Model of the Relation between White and Yolk Weights of Birds' 

Eggs. 

G. A. Baker, University of California, Davis. 

Statistical Analysis for a New Procedure in Sensitivity Experiments. 

W. J. Dixon, University of Oregon, and A. M. Mood, Iowa State College. 

The Relation of Inbreeding to Calf Mortality. 

P. W. Gregory, University of California, Davis. 

Cooperative Field Trials. 

P. A. Minges, University of California, Davis. 

Population Genetics. 

N. H. Horowitz, California Institute of Technology. 

Statistical Problems in Assessing Methods of Medical Diagnosis^ with Particular 

Reference to X-Ray-Technique. 

J. Yerushalmy, United States Public Health Service, Washington, D. C. 

Discussion: J. Neyman, University of California, Berkeley. 

Professor John W. Miles was chairman of the afternoon session on June 18, 
which was a joint session ^vith the California Section of the American Society 
for Quality Control. The following papers were presented: 
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Topic: Industrial Applications of Stalistics. 

Papers: Operating Characteristics of Average and Range Charts. 

Henry Scheff4, University of California, Los Angeles. 

Sampling Inspection by Variables. 

Herbert Solomon, Stanford University. 

Some Exact Numerical Results for Sequential Acceptance Sampling by Attributes. 
Mark W. Eudey, University of California, Berkeley. 

Choice of Inspection Stringence in Acceptance Sampling by Attributes. 

Joseph L. Hodges, University of California, Berkeley. 

Widening Tolerances for Closer Fitting Parts. 

Edmond E. Bates, Quality Engineering Consultants, Los Angeles. 
Discussion: Russell O’Neill, University of California, Los Angeles. 
Re-establishing Operator Responsibility for Quality Control. 

Wyatt H. Lewis, General Electric Company, Ontario, California. 

Discussion: William B. Rice, Plomb Tool Company, Los Angeles. 

The Application of Learning Curves to Industrial Planning. 

James R. Crawford, Wright Field, Dayton, Ohio. 

The Wednesday evening session was under the chairmanship of Professor 
George Beadle, California Institute of Technology, with the following program: 

Topic: Statistical Problems in Genetical Studies in Chickens. 

Paper: Rate of Genetic Gain in Egg Production in Progeny-tested Flocks as a Function of 
the Interval between Generations. 

Everett R. Dempster and I. Michael Lerner, University of California, Berkeley* 

On Thursday morning, June 19, there was a joint session ^\^th the Western 
Psychological Association. Professor Helen Walker of Columbia University was 
chairman. The program was as follows: 

Topic: Statistical Problems in Psychology. 

Papers: Statistical Criteria of the Effectiveness of Selective Procedures. 

R. F. Jarrett, University of California, Berkeley. 

Unsolved Statistical Problems Arising in Psychological Measurements. 

Helen Walker, Columbia University. 

Cost Utility Curves as a Means of Assessing Batteries of Tests. 

Joseph Berkson, Mayo Clinic. 

Approaches to Univocal Factor Scores. 

J. P. Guilford, University of Southern California. 

The afternoon session on June 19 was under the chairmanship of Professor 
Harald CramtT of Stockholm, Sweden, and offered the following program: 

Topic: Theory of Statistics and its Applications to Astronomy. 

Papers: Random Variables with Comparable Peakedness. 

Z. W. Birnbaum, University of Washington. 

Distributions which Lead to Regressions Representable by Polynomials. 

Evelyn Fix, University of California, Berkeley. 

Optimum Tests of Composite Hypotheses with One Constraint. 

Erich L. Lehmann, University of California, Berkeley. 

Estimation of a Distribution Function by Confidence Limits. 

Frank J. Massey, Jr., University of California, Berkeley. 

A Note on Sequential Confidence Sets. 

Charles Stein, Columbia University. 
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Certain Types of Statistical Problems in Astronomy. 

Robert J. Trumpler, University of California, Berkeley. 

Basic Concepts of the Theory of Statistics in Relation to Certain Problems of 
Astronomy. 

J. Neyman, University of California, Berkeley. 

A Note on the Problem of Binary Stars. 

Elizabeth L. Scott, University of California, Berkeley. 

Explicit Solution of the Problem of Fitting a Straight Line when both Variables 
are Subject to Error for the Case of Unequal Weights. (By title) 

Elizabeth L. Scott, University of California, Berkeley. 

Power Function of the Analysis of Variance and Covariance Test of a Normal 
Bivariate Population. (By title) 

Way Ming Chen, University of California, Berkeley. 

Unbiased Estimates with Minimum Variance. (By title) 

Charles Stein, Columbia University. 

Sufficient Statistics and a System of Partial Differential Equations. (By title) 
Erich L. Lehmann, University of California, Berkeley, and Henry Scheff4, 
University of California, Los Angeles. 


On Wednesday evening, June 18, at 6 o’clock, there was a dinner for members 
and guests, at the Hotel San Diego. 


P. S. DWYER 
Secretary 



ON OPTIMUM TESTS OF COMPOSITE HYPOTHESES WITH ONE 

CONSTRAINT^ 

By E. L. Lehmann 
University of California^ Berkeley 

Summary. This paper is concerned with optimum tests of certain composite 
hypotheses. In section 2 various aspects of a theorem of Scheff^ concerning type 
Bi tests are discussed. It is pointed out that the theorem can be extended to 
cover uniformly most powerful tests against a one-sided set of alternatives. 
It is also shown that the method for determining explicitly the optimum test 
region may in certain cases be reduced to a simple formal procedure. These 
results are used in section 3 to obtain optimum tests for the composite hypothesis 
specifying the value of the circular serial correlation coefficient in a normal 
distribution. A surprising feature of this example is the fact that for the simple 
hypothesis obtained by specifying values for the nuisance parameters no test 
with the corresponding optimum properties exists. 

In section 4 the totality of similar regions is obtained for a large class of prob¬ 
ability laws which admit a sufficient statistic. Some composite hypotheses 
concerning exponential and rectangular distributions are treated in section 5. 
It is proved that the likelihood ratio tests of these hypotheses have various op¬ 
timum properties. 

1. Introduction. In developing tests for a class of hypotheses three phases 
may be distinguished. First, tests are obtained which are intuitively appealing; 
next, it is shown that these tests have certain attractive features; finally, it is 
proved that they are ‘‘best possible” tests. 

In dealing with parametric hypotheses, the likelihood ratio principle is fre¬ 
quently used to obtain a reasonable test. For many of the tests so derived for 
normal and exponential distributions, the question of bias has been investigated. 
In most cases unbiasedness has been established; in the other cases, usually a test 
based on the same criterion but with the boundaries shifted, can be proved to be 
imbiased. Other desirable properties which likelihood ratio tests have been 
shown to possess, relate to the asymptotic behaviour of these tests as the sample 
sizes tend to infinity. An interesting problem which does not seem to have been 
treated is the question of admissibility of likelihood ratio tests, a test being ad¬ 
missible if its power can not be improved upon uniformly by any other test of 
the same level of significance. 

Investigations of optimum tests of composite hypotheses have been carried 
through for many hypotheses concerning normal distributions. When the hy¬ 
pothesis specifies the value of one parameter (hypothesis with one constraint), 
uniformly most powerful one-sided and type Bi (uniformly most powerful un- 

^ Presented at a meeting of the Institute of Mathematical Statistics in San Diego, June, 
1947. 
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biased) tests have been obtained. When the number of constraints is larger 
than one, not so much can be expected. It has been shown for some of the tests 
in this class that they have maximum average power uniformly over a family of 
surfaces in the parameter space, or that they are uniformly most powerful with 
respect to the subclass of testa whose power depends only on some function of 
the parameters. (All optimum properties mentioned are relative only to the 
class of all similar regions. This will be so throughout the paper and will usually 
not be stated explicitly). 

Two methods for finding uniformly most powerful or uniformly most powerful 
one-sided regions and type Bi tests, if they exist are known. Neyman and Pear¬ 
son [1] developed a method for determining all similar regions, and applied it 
to obtain uniformly most powerful one-sided tests of certain hypotheses. Ney- 
man [2,3] extended the method to obtain, for certain hypotheses, the class of all 
bisimilar (unbiased similar) regions, and Scheff^ [4], developing the method 
further, proved the existence of type Bi tests for an important class of hypotheses. 

A different method for obtaining all similar and bisimilar regions was devised 
by P. L. Hsu and was used by him and other writers to prove various optimum 
properties of the likelihood ratio tests for the general linear hypothesis, of Hotel¬ 
ling's and of other tests [5, 6, 7, 8]. 

In the present paper we are concerned with applications of these two methods 
to composite hypotheses with one constraint. However, the applicability is not 
so restricted. In fact, the second method has been used mainly in connection 
with composite hypotheses with many constraints, and the author believes it to 
be suitable also for deriving optimum classification procedures. An essential 
restriction of both methods seems to be that a set of sufficient statistics must exist 
with respect to the parameters involved: with respect to the nuisance parameters 
so that all similar regions can be found, with respect to the parameters specified 
by the hypothesis so that there exists a best of all similar regions. 

Extensions of the existing theory based on the first method are obtained in 
section 2, and the theory is applied in section 3 to a hypothesis concerning a mul¬ 
tivariate normal distribution. Sections 4 and 5 are concerned with applications 
of the second method to problems to most of which the earlier method is not 
applicable, in particular to hypotheses concerning exponential and rectangular 
distributions, hitherto only treated from the likelihood ratio point of view. 

2. On the theory of optimum tests. 

2.1 0ne-9ided tests. In an interesting paper [4], Scheff4 determined the type 
B and type Bi tests of a certain class of composite hypotheses specifying the 
value do of a parameter 6 in the presence of nuisance parameters. 

Scheff^'s results can, in an obvious way, be extended to cover one-sided sets 
of alternatives. To show this, consider the method used in [4]. Under certain 

assumptions all tests* are found which satisfy the two conditions: 

, 

* The terms **the test w** and **the region [of rejection] w will be used interchangeably. 
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(a) The power function /S* at has a preassigned value € (the level of signifi¬ 
cance), independent of the nuisance parameters; 

(b) the power function at So has derivative 0. (Condition of unbiasedness). 
Then that test Wo is determined for which, of all those satisfying (a) and (b), 

(c) the second derivative at So , pZ(So)j is as large as possible. 

By definition wo is a type B test. Under a certain additional assumption (this 

d^g • 

is the convexity assumption r-j > 0 of Scheff^’s Theorem 2) it is shown that of all 

oyi 

tests satisfying (a) and (b), wo has maximum power against all alternatives, 
i.e. is of type Bi . 

If now we want to maximize the power against only the one-sided set of alter¬ 
natives, B > So y we determine that test Wi of all those satisfying (a), for which 

(d) the first derivative at , ^LiSo), is as large as possible. 

Under a certain additional assumption (in Scheff6*8 notation this would be the 
Sg 

monotonicity assumption — > 0) it can then be shown that of all tests satisfy- 
oyi 

ing (a), Wi has maximum power against all alternatives S > So ^ (it also has 
minimum power against all alternatives S < S^, i.e. Wi is uniformly most power¬ 
ful against alternatives S > So ^ We shall not carry through the discussion 
in detail since Scheff^’s argument applies step by step, with only the obvious 
changes. 

2.2 Determination of the boundaries. Let Xi, • • • , Xn be n random variables 
with a joint probability density function p, depending on parameters Si and S = 
(^ 2 , • • • , 0f). We shall denote the probability density function of a set of ran¬ 
dom variables Xi, • • • , Xn whose distribution depends on a parameter S by 
p(xi , • • • , Xn\S) or simply by p{xi , • • • , a^n) when the dependence on S is 
clear from the context. The set of points {xi, • • • , x„) for which 

pfe , • • • , Xn I e) 

is positive we shall denote by W+{S), 

Let 


(2.1) <o.(Xi, • • •, x„) = ~ log p(xi, • • •, x„ I , 9) , (i = 1, • • •, 1), 

and let the random variable be defined by 

(2.2) =.^.(Xi,-..,X„). 

Then for testing the hypothesis H\ Si = s\, under the assumptions stated by 
Scheff6, the type Bi test Wo is defined by the inequalities 

(2.S) ^ , > fe {ki < h%) 

where h , fe depend on , • • • , and are determined by the two equa¬ 

tions* 

(2.4) [ iplp(<pi, - • y<pt) dipi = (1 - €) f same (s == 0, 1). 

Jki •*-00 

• Although k\ and kt may depend on wo is independent of as was shown, in [4]. 
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The equations (2.3) and (2.4) are not suitable for the determination of the 
boundary of . The variables have to be transformed so as to obtain for 
an expression from which the calculation of the boundaries becomes feasible, 
(cf. [9]). This part of the work may be formalized in the following theorem. 
Theorem 1 . Let 


(2.5) 


V = /(^i, 4>2, • • • , 4>i) 


Vi = fif,(4>2, ••• ,<!»/), 


(^ = 2, ... , 0, 


be a system of functions, continuously differentiable and with non-vanishing Jaco¬ 
bian almost everywhere, and such that 

(i) U is a linear function of 

(2.6) ^ U = a^i + b 

with coefficients which may depend on 4>2, • • • , and such that^ a{^ , • • • , 4>/) > 0» 

(ii) it is possible to solve /or 4>2, • • • ,^i in terms of the F’s, 

(iii) under the hypothesis II, U is distributed independently of 

7= (72,---,F0. 


Then the region Wa is equivalent to the region 
(2.7) u < Cl, > ct (ci < C 2 ) 


where Ci, C 2 are determined by 

( 2 . 8 ) 

Proof. 

P(^I , ^i) = P(W, Vi, • • • j Vi) 


u*p(u) du = (1 — e) f u*p{u) du {s =* 0, 1). 

Cl J-QO 

diu, V2, 


(2.9) 


d{ipi, ••• , fpi) 

du 


p(u)-p(V2, ’ - yVl) 


dipi 


d(vi, 

••• . 0 |) 

d{<p2, 

... ,<pt) 


But 

( 2 . 10 ) 


( 2 . 11 ) 


U = 0(v?2 f ••• 1 <pl) • fpl + 1>{¥>2 y * ifPl) 

= a(V2 , • • • f Vi) • <Pi + fi(V2 , • • • fVi) 

so that (2.4) reduces to 

\ a / 

= (1 — €) / same (s « 0, 1) 

J—90 


* A similar theorem holds when we assume a(^ , .. ., 4>*) < 0. 
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and hence to 

( 2 . 12 ) 


u*p{u) du = (I — e) 

Cl(Vi,-”,Vl) J—« 


same 


(S = 0, 1) 


which shows Cj and C 2 to be independent of the v^s. Also obviously (2.3) trans¬ 
forms into (2.7) which completes the proof. 

If U is such that its distribution (when 0i = 0?) is independent of Ci and C 2 
of theorem 1 will depend only on the data of the problem: c, n, 0?. However, the 
existence of constants Ci and C 2 satisfying (2.8) still has to be proved. We may 
show more generally the existence of ki and k 2 satisfying (2.4). A proof is im¬ 
mediately supplied by an argument which was used by Ne 5 rman [10] and Wald 
[11] to prove the existence of type A tests, and which may be stated in the 
following 


Lemma. 


LetO < a < ljletf(x) > Qand 



dx < 00 /or s = 0,1. 


Then 


there exist A, B such that 


(2.13) 



is = 0, 1). 


3. Testing for circular serial correlation in a normal population. We now 

apply the results of the previous section to obtain the optimum tests (i.e. uni¬ 
formly most powerful against the one-sided set of alternatives, type Bi in the 
two-sided case) for the hypothesis specifying the value of the circular serial cor¬ 
relation coefficient in the normal population considered by Dixon [12], (For 
the literature on testing for non-circular serial correlation in normal populations 
cf. [12]). 

We assume 

1 — Jn r n -1 

(3.1) p(xi, • • •, x„) = ~ ^ S 

where Xn+i = Xi and \ 8\ < 1, and we test the hypothesis 5 = 5o. For testing 
purposes only the value 5o = 0 is of interest presumably, however, the family of 
tests for arbitrary 8q is required for estimating 8 by means of confidence intervals, 
and therefore the more general hypothesis is considered. 

Making a transformation in one of the parameters we write 

, • • • , Xn) 

= C{s, a) expj^ a 1^(1 + «’) Z (Xi - {)* “ - {)(*m - {) 

where in the notation of the previous section di = 5, 02 = a, 08 = f. 

Theorem 2. For testing the hypothesis 5 == 0o for the distrihtUion (3.2) 

(a) the type Bi test exists and is given by 

(3.3) 


r < ri, > r 2 
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where 


(3.4) 


r 


« 

E 


(Xi x) (^t+i x) 
J2 (Xi - xf 


X = - T.Xi 

n i^i 


and where n and r 2 are determined by 

(3.5) ^ (i + 2aor) = (1 - «) same (s = 0,1). 

(b) the uniformly most powerful similar region for testing H against the alter¬ 
natives 5 > 5o exists and is given hy 

(3.6) r > r' 


where r' is determined hy 

(3.7) f p(r) dr = (1 — e) f p(r) dr} 

00 00 


Proof. We compute 

ipi = Ci(5o, a) + 2a[5oS(x»* — {)* — S(x< — f)(aJi+i f)l 

(3.8) (f>2 = ,«) + (! + 5o)S(Xi — iff — 28o^(Xi — f)(x*-+i — f) 

^ — 2na(l — 5o)(iC — f). 

There is no difficulty in checking the conditions of Scheff^’s theorems [4], 

Next we apply Theorem 1 of the previous section, and define 

F 2 = (1 + Sl)2(X, - Xy - 25oS(X, - - X) 

(3.9) y, = X - f 

,, ^ 2(X, - X)(X,4i - X) 

V 2 

Conditions (i) and (ii) of Theorem 1 are easily seen to be satisfied. To show that 
U is independent of 7 = (72, 73 ) we employ arguments which have recently 
been used by various authors in a number of similar problems (cf. [13,14,16]). 

It is seen that an orthonormal transformation exists: 



Xi, 



such that 






•\/nX 

= Yr 

(3.10) 

E(x* 

^-1 

- - X) 

= Ex,y? 



E (Xi - xf 

= Ef!. 



<-i 

<-2 

‘ A corresponding result holds 

for the other one-sided case. 



OPTIMUM TESTS 


479 


Under H the Y*& are distributed with probability density 

]] 

where kffia, • • • depend on 5o and where the m's are all positive. Introducing 
new variables 

(3.12) a = 2. •••, n), 

and, then, generalized polar coordinates in the space of the Z% 


(3.11) p(yi, ••• ,^n) = C'(8o, a) exp j^a j^fc(yi - Vni)* + 2 wV* 


(3.13) R — ^ ^ Zif > '^u -2 

we see that Vi, R and , • • • , S^n -^2 are completely independent. Also 
7, = /2», r, = ^(n-{) 

while U, being homogeneous of degree 0 in the Z% is a function of the S^^^s only. 
This proves that U, V 2 and V 3 are completely independent. The type Bi test 
of H is therefore given by 


(3.14) u 


23 (^1 ^)C^i+l ^) 

---f=l--- <C„>C 

(1 + Jo) S ~ ®)* ~ 25o (Xi — x)(X{+i — £) 

•-1 t-l 


where Ci and C 2 are determined by 

(3.15) f u^p(u) du = (1 — e) f u^puQ du (s « 0, 1). 

Jei J —00 

We still have to show that this test is equivalent to the one defined by (3.3) 
and (3.5). For So = 0 this is trivial. Let us assume So < 0. (The other 
case goes through similarly.) The inequality w < Ci is equivalent to 

(3.16) (1 + 2S,fii)2(x< - x)(xi+i -»)<(!+ a?)2(a;< - x)’ 


and hence to 


(3.17) 

provided 1 + 2ci8o > 0. 


Z(Xi - g)(zw - £) 

S(Xi - £y ‘ 

Suppose 1 + 2ciJo < 0, i.e. ci > — . 

aOo 


Then* 


(3.18) 


P {U < c,} > p|c7 < -J-J = P{0 < 2 (Xi - X)*} = 1 


• We denote the probability of an event A by F 
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i.e. P{U < Cl] =1 which would contradict (3.15). Similarly if 1 + 2c2do < 0 
we would have P{ (7 > C 2 ) = 0 and hence our test would be one-sided and there¬ 
fore not unbiased. The inequalities m < Ci, > C 2 are thus equivalent to the 
inequalities r < n , > and since 

r 

1 -{- 60 — 26or ' 

(3.5) also follows. 

The existence of type Bi and uniformly most powerful one-sided tests of the 
hypothesis H is rather surprising. For when a and { are assumed known, neither 
the type Ai test nor the uniformly most powerful one-sided test of the simple 
hypothesis //': 5 = 5o exists. This is easily seen by determining the most 
powerful and the most powerful unbiased test against a specific alternative 5i 
for the hypothesis //' in the population 

1—5" 

(3.19) v{xi, •• • jXn) = [~M(l + 5^)2a;? - 25Sx<a;»+i]]. 

The distribution of the criterion R was obtained by R. L. Anderson [16] (see 
also [17]) for the case 5 = 0. Madow [15] using Anderson^s result found the dis¬ 
tribution for arbitrary 5. (Approximations to the distribution have been studied 
by various authors; for the literature on this rf. [18]. Recently Hsu [19] ob¬ 
tained an asymptotic expansion.) A direct derivation for arbitrary 5 may be 
based on the following theorem of Cramer, which was communicated to the 
author by Dr. P. L. Hsu. 

Theorem 3. {Cram4ry, If X, Y are two random variables^ {not necessarily 
independent) y F > 0, then 

(3.20) 


where <px cmd ^ are the characteristic functions of X — xY and Y respectively, 
provided 


( 3 . 21 ) 



<pAt) - m 

t 


dt < QC . 


Theorem 4. If 


( 3 . 22 ) 


P(*I , • • • , Xn) 


1 — 



[(»< - & - «(x<+l 



(a^,+i = xi) 


’ Differentiated forms of the theorem were given by R. C. Geary [Jour. Roy. Stat. Soe. 
Vol. 107 (1944) p. 661 and H. Cram4r [Exercise 6 on p. 317 of Mathematical Methods of 
Statistics. Princeton Univ. Press (1946)]. 
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and if 

(3.23) 

Oien 

(3.24) 


Z {Xi - - X) 


P{R > r) = 




on+1/2 


Z (Xi - X^) 

t-l 


1 - 8 ^ 


n (I - 5)(1 + 5' ~ 25r) 

(-1)'+‘(X,- -r)"-«*sm^^sin?^ 


•E 


n n 


1 + 62 - 26X/ 


tt^/iere summation is extended over all integer 1 < j < -, for which Xy > r, and 

JU 

where 


(3.25) X, = 2cos^. 

n 

The proof of this theorem from Theorem 3 is straightforward and only will 
be indicated here. If X and Y denote the numerator and denominator of R 
respectively, the characteristic functions of Y and X — rY may be obtained by 
the method of circulants (cf, [12,17]). The integral on the right hand side of 
(3.20) is then easily evaluated by’the theory of residues when n is odd. In the 
case that n is even, the integrand has two branchpoints, one in the lower and one 
in the upper half plane. These may be separated, and then again the method 
of residues may be applied. 


4. Similar regions. The problem of finding all regions similar to the sample 
space with respect to a parameter 6 was solved by Neyman and Pearson [1] for 
a certain class of probability laws. In a later paper Neyman proved ([20] 
proposition IX) that if there exists a sufficient statistic T for a parameter 
then w is similar with respect to 6 if it has the following structure: For the inter¬ 
section w{t) of w with the surface T = t, the relative probability of w{t) given 
T = t has a constant value independent of L We shall show in this section that 
for a large class of probability laws which admit a sufficient statistic for 6 the 
regions with the above structure are the only ones that are similar with respect 
to e. 

We consider samples from a univariate distribution and we distinguish three 
cases as one, both or neither of the extremes of the range of the distribution 
depend on the parameter 6. For the first of these cases (cf. Pitman [21]) we con¬ 
sider samples from a distribution with probability density 
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(4.1) p(x) = ^ . He) <X < C, 

where k{6) is a strictly monotone continuous function of $ and where c may be 
infinite. Introducing a new parameter S = k(d) the distribution of a sample 
from (4.1) is given by 

(4.2) p(x.,---,x„) S<Xi<c. 

To obtain the totality of regions w similar with respect to 6 let us denote by 
Wi, • • • , Wn the portions of the sample space where the smallest of the x*a is 
Xi, • , Xn respectively. For any region w denote by wl the intersection of w 

with Wk . Consider a transformation carrying W 2 , • • • , Wn into Wi , letting 
yi = min(xi, • • • , a;n) and letting in Wk : 


(4.3) 2/2 = xi, yi = X2y ••• f yk = Xk-i , yk+i = Xk+i y ••• , yn = Xn- 

Denote by Wk the image of Wk under this transformation. The condition that 
w be similar with respect to 6, 


(4.4) 


/. 


f(Xi) ■ • ■ f(x„) 
b{B) 


dxi • • • dXn 


( 5 ) 


may be written in the form 

r ® {§/. 


(4.5) 


JufkiVi) 


rfy.j dyi 


f(yn) dyt 

= n« / { f ' f(yi) ’ • • f(yn) dyi ••• dpA dyi 


where W{y^ denotes the region yi < yi < c, (t = 2, • • • , n), that is, the region 
of variation of 2 / 2 , • • • , 2/n given 2/1 , and Avhere Wk{y^ denotes the region of vari¬ 
ation of 2 / 2 , • ■ • yVn given 2/1 and Wk . From (4.6) we obtain 

(4.6) ^ dyi s 0 

where 


^( 2 / 1 ) “ Z f /(2/2) • • • /(yn) d2/2 • • • dyn 

k^l JwkivO 

- nt f ••• f /(ys) •••f(yn)dyf”dyn. 

. •'t/I •'Vi 


(4.7) 

But (4.6) implies 


(4.8) 


^( 2 / 1 ) = 0 almost everywhere 


and since we can only determine up to a set of measure 0, we may omit the 
qualification in (4.8). Therefore a necessary and sufficient condition for w to 
be similar is 
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(4.9) 


n /(y) dyj 


n » 

pi S / f(yt) • • • /(yn) dys • • • dj/n - « 


for all . 

To see more clearly the structure of these regions, let us take n = 2. Equa¬ 
tion (4.9) states that on each of the broken lines of Fig. 1 the relative probability 
oiw = wi + wi given Yi = yi is €, where the decomposition of this probability 
into its two components may vary with yi . 





In general equation (4.9) states that on each hyperplane Yi = yi the relative 
probability of w is independent of yi , Since Yi = min {Xi , • • • , Xn) is a suffi¬ 
cient statistic for 6 , Neyman’s theorem in this case does give all similar regions. 

Next let us consider the case where both extremes of the range of the distribu¬ 
tion depend on the parameter. We shall assume (cf. 121]) that , • • • , 
are distributed with probability density 

( 4 . 10 ) “ e<x<m 

where 5 is a strictly decreasing continuous function over an interval [—oo, 
6(—oo)] and where 6[6(—<»)] = — oo. These assumptions insure that there 
exists a unique number a, — «> < a < 6(— «>), such that 6(a) = o. 

Denote by , (i, j = 1, • • • , n; i j), the portion of the sample space 
where the smallest and the largest of the a;’s are Xi and respectively. Denote 
by Wiji and Wm those portions of Wij where Xi is greater than and less than 
6“^(a;y) respectively. For any region w denote by tajy* the intersection of w with 
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Wijk . Consider a transformation carrying the sample-space into Win , letting 
2/1 = min (a;i, • • • , Xn), Vn = max (a;i, • • • , Xn) and in letting 2 / 2 , • • * , 2/n-i 
denote the remaining x's in the order of their subscripts. Next make a trans¬ 
formation carrying Wm into Wini , letting Zi = max [yi, h~\yn)]y Zn = min 
[ 2 / 1 , h~^(yn)] and Zk = ynioT k = 2, ,n — 1. Denote by Wijk the image of 
Wijk in TFini. 

Then Zn is a sufficient statistic for B (cf. [21]) and there exist functions/i, gi 
such that the density of Zn is given by 


(4.11) 


X)(z ) = 

gm 


in 


g Zn ^ a 


while the distribution of the remaining Z*s given Zn is independent of B. 

The condition that who a similar region may now be written, analogously to 
(4.5), in the form 


(4.12) 


fVl(gn) ^ f 

QliO) i,j,k ''wij, 


*o<ik(*n) 


P(2l 


, z„_i ( Zn) dzi 


dz, 


n-rdZn = ^ j/^^Uzn 


and hence by the argument which led to (4.6), as 


(4.13) S / p(zi , • • •, z„_i I fcn) dzi-- - dzn-\ = « for all z„ . 

i,j,k Ju>ijk(Mn) 

Thus in this case also Neyman’s theorem gives the most general similar region. 

For the case that neither extreme of the range of the distribution depends on 
the parameter 0, it has been shown by various authors [22, 21, 23] under slightly 
varying assumptions concerning the regularity of the distribution function, that 
the existence of a sufficient statistic implies 

(4.14) pix I B) = exp [P(^) -f T{x)Q{B) + /?(a;)]. 

This (cf. [10]) is a special case of that for which Neyman and Pearson determined 
the totality of similar regions, however under the restriction that the moments 

of $ = ~ log p{X^ uniquely determine the distribution of <&. We shall 
dB t’.i 

f 

briefly indicate how this assumption may be avoided. 

Let Xiy • • • , Zn be a sample from (4.14), or, more generally, (this is the case 
considered by Neyman and Pearson), let Zi, • • • , Zn be distributed with prob¬ 
ability density 

V{xi, ,Xn) 

(4.15) 

= exp [P(0) + uiXi , • • • , Xn)Q{6) -f v{Xi , • • • , Xn)] 

in a sample space W+ which is independent of B. We shall assume that the set 
of values which Q takes on contains at least some interval. Introducing 6 == 
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—Q(0) as a new parameter, we shall obtain all regions similar to 5 (where the 
set of values of 5 contains an interval) for the distribution* 


(4.16) 


p(X] 


1 1 


,Xn) 


= exp [pi(5) - 5 • u(xi , 


Xn) + v(X: 


1 > 


Xn)] 


under the assumption that 



0 except possibly on a set of measure 0. 


Let us for a moment assume that there exist functions /t(x<, ••• , Xn), 
(z = 2, • • • , n), with continuous partial derivatives almost everywhere and such 
that the transformation 


(4.17) 2/l u{Xl I * * * , 3Jn) J 2/* — /t(^l > * * * » ^n)) — 2, • • • , ?i), 

is one to one on except possibly on a subset of measure 0. Applying this 
transformation we may write the condition of similarity in the form 

“ f Vn) dy,--- dy.-dy, 

= € /■ “/ /(y,, • • • , y.) dy* • • • dy^-dy, 

J-oo JwivO 


L 

(4.18) 


where W{y^ denotes the region of variation of 2 / 2 , • • • , 2/n given y \, and where 
ty( 2 /i) denotes the region of variation of 2 / 2 , • • * , 2/n given 2/1 and w. Furthermore 
AVi , • * * , 2/n) is independent of 5. From the theory of bilateral Laplace trans¬ 
forms it is known that (4.18) implies that 


(4.19) / /( 2 / 1 , • • • , yn) dy 2 ---dyn = € f(yi, , yn) dy 2 • ^ dy^ 

•'w(Vi) 

which is the desired result. 

More generally it may be shown that our assumption concerning u{xi , • • • , a^n) 
insures the existence of functions/*, (i = 2, • • • , n), such that under the trans¬ 
formation (4.17) no point ( 2/1 • , yj has more than a denumerable infinity of 

counter images in a;-space. Our proof can be modified to cover this case. The 
argument is similar to that used to obtain equations (4.9) and (4.13) which were 
also arrived at through many to one transformations. 


5. Testing exponential and rectangular distributions. In their fundamental 
1928 paper [24] on likelihood ratio tests, Neyman and Pearson discussed various 
hypotheses relating to normal, exponential and rectangular distributions. Later 
they and other authors developed a theory of similar and bisimilar regions which 
made it possible to obtain optimum tests of many composite hypotheses with 

® An assumption that wo can solve for 0 as a function of 5 is not needed since we can 
determine Pi («) by integrating the density (4.16) over 
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one constraint concerning normal populations. This theory however is not 
applicable to most hypotheses concerning exponential or rectangular distribu¬ 
tions. We shall in this section obtain optimum tests of some hypotheses relating 
to these latter distributions, using the method of the previous section. 

Let us first consider a sample , • • • , Xn from an exponential population, 
the probability density of the sample being. 

(5.1) ?(*!, ••• ,X„) = iexpf(r< - 6)1 ifa-<>6, (i=l, •••,n) 

O" L O i-l J 

and let us consider the two hypotheses H\:a =o, U 2 ib = bo where, without loss 
of generality, we shall take Oo = 1, fco = 0. The likelihood ratio tests of both 
these hypotheses were shown to be completely unbiased by Paulson [25]. We 
shall prove 

Theorem 5. The likelihood ratio tests of Hi and Ih are type Bi and uniformly 
most powerful, respectively. The one-sided tests based on the likelihood ratio criterion 
for Hi are the uniformly most powerful one-sided similar regions for testing this 
hypothesis. 

Proof. In order to simplify the argument we shall give a detailed proof only 
for the restricted class of tests which are symmetric in the variables Xi, • • • , Xn . 

For testing Hi let us make the following transformation introduced by 
Sukhatme [26]: 


( 6 . 2 ) 


Zi = nYi 
Z, = (n - i + i)(r, - 

where is the ith of the X*8 in order of magnitude. Then 

p(2i, • • • , 2.) = ;^ exp (zi - n6) - ^ 23 

o" L o o <_8 J 


(i = 2, • • • , n), 


(6.3) 


if Zi > nb;Zi > 0 (^ = 2, • • • , n). 


We want to determine all regions w which under H are similar, to the sample 
space with respect to 6, i.e. all regions w satisfying 

j exp ^ C?S 2 • • • dZn dZi 

= f / f exp r — ctej • • • d2n| dzi 

Jnb L »—2 J ) 


(6.4) 


(b) (b) 

S € 


s « / e 
Jnb 


♦• 6 ) 


dzi 


where w{z^ denotes the intersection of w with the hyperplane Zi = . Now 

(5.4) is equivalent to 



( 6 . 6 ) 

where 
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f. 

Jnt 


<») 

<fei a 0 


(5.6) 


/(2l) •= f exp r - 52 <fea • • • <fen - 

L 2 J 


and this in turn is equivalent to 

(5.7) f(zi) = 0 for all Zi . 

Of all the regions w satisfying (5.7) we want to determine the one which against 
a specific alternative, say ai , has maximum power, i.e. for which 


(5.8) 


/■ 


-{Vai){Mi-nb) 


f exp — - ^ zA dz 2 • • • dZn dzi 
L *-2 J 


is as large as possible. We thus see that w will have the desired properties if 
w( 2 i) is determined according to the two conditions 


(5.9) 
and 

(5.10) 


/ exp — 52 s< dzt ••• <fe« 
L •—2 J 

f exp — - 53 • • • d«n = 

^(* l ) L Ol •-2 J 


max. 


Hence by the Neyman-Pearson fimdamental lemma wizi) is the set of points 
satisfying 

(5.11) exp r(- ^ C(ai, zi) 

L\ »-2 M / J 

and therefore according as ai is greater or less than 1, w(zi) is determined by 


(5.12) 


Sz* = - min (a:i, •••, *»)) > Hou *i). or 

<-2 <-l 


n n 


53z< = 2[xi — min (xj, ••• , x„)] 

M s-1 


< 


Aj'( 0 i , Zi). 


n 

But 53 is independently distributed of Zi and under H the distribution of 

imd 

n 

53 Zi does not depend on ai, in fact it is a chi-square distribution with 2n — 2 

degrees of freedom. Thus k and k', as determined by (5.9) are independent of 
Oi and the two tests (5.12) are uniformly most powerful one-sided. 

Next we consider the more restricted class of unbiased similar regions. For to 
to be unbiased we must have 
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^[ - Is ^’'} I 

(5.13) = / (^1 — n5 — n) exp [— (21 — nh)] / exp — XI 2 ,- (^22 • • • <?2n dzi 

•'nb •'w(ri) L »■—2 J 

+ / exp [ — (21 — nb)] / ( ) exp — X) 2,* J 22 • • • cfen dzi = 0. 

•'nb \i-«2 / L «—2 J 

The first of the integrals in the middle member equals 

/ (2 — n) e“* / exp — X^ 2*- 6/22 • • • d2n d 2 

•'O •'tt)(*+n5) L <—2 J 

- € (2 — n) e“* d 2 = —(n — 1)€. 

Jo 

Therefore 

r f (E exp r - E ffe 2 •••(&„ ^/ 2 , 

Jnb JwiMi) \t-2 / L t'-2 J 

= (n - 1)* = (n - 1)6 f rfji 

Jnb 


(5.14) 


(6.15) 


or 

(5.16) 

where 


r«o (b) 

/ < 7(20 rfzi = 0 

Jnb 


(5.17) sr( 2 i) = [ exp j - X^2*] ^22 •• • dzn ~ (n — 1)6. 

\t«2 / L *'-2 J 

Thus finally the condition of unbiasedness reduces to 

(5.18) / ( 5 ^ 24 ) exp — X} 2i d 22 • • • dzn = (n — 1)6 

•'»(*!) \*-2 / L *-2 J 

and we seek the region w{z^ which satisfies (5.9), (5.10) and (5.18). 

By the fundamental lemma w{z^ is given by 

(5.19) exp > 1^(71(01, zOE*.- + C' 2 (ai, zi)j • exp E®*^ 

which is equivalent to * 

n 

(6.20) E . 2l), > fe(0i, ii) 

i-2 


where ki and fe are determined by (5.9) and (5.18), and are therefore independent 
of Zi and a. Thus the region (5.20) which of all imbiased similar regions 
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maximizes the power against the alternative a = ai is independent of ai and 
hence is a region of type Bi . This completes the proof since it is easily verified 
that (5.10) is equivalent to the likelihood ratio test. 

The proof for regions which are not necessarily S 3 unmetric in the variables 
follows similarly if instead of the transformation (5.2) one uses a transformation 

n 

Ui = fi{Xi , • • • , Xn) which is one to one and such that Ui = Zi, U 2 = ^ Zi. 

t —2 

The distribution of t/ 3 , • • • , f/« is then independent of a and b since Ui , U 2 
are a pair of sufficient statistics for these parameters, and the proof carries over 
step by step. 

Next we consider the hypothesis H 2 :h = 0, and again we restrict ourselves to 
regions which are symmetric in the variables, although as before the proof can 
be modified to cover also nonsymmetric regions. 

We first make the transformation to Zi, • • • , given by ( 5 . 2 ). In the 
n — 1 dimensional space of Z 2 , • • • , , we then transform to new variables 

n 

f/, , • • • , ^n -2 where U = ^ Zi and where the are the generalized polar 

t -2 

angles. Obviously the distribution of the ^'s does not depend on a, since they 
are homogeneous of degree 0 in the Z’s. Furthermore the ^’s are independently 
distributed of U since the probability density of the Z*s is constant over the 
hyperplanes U = u. Thus 

p{Zl, n, ^ 1 , • • • , ^„-2) = “n 

^n -2 ^ ^n- 2 ). 

We next introduce new variables 


[ zi — nh 
~ —^1 


(5.22) F = 2^1 + C/ and T = 
and find 

» 

p(v, , 'pn-i) = ^ ®^p [ ~ (1 - piPu 

(5.23) 

for V > nh, — < i < 1. 

V ~ 

For v) under Hi to be similar with respect to o, we must have 

J ^expF- ^1 (1 ~ • • • ,^n- 2 )dtdifi • • • d^nr-i • dv 

r "I 
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where wiv) designates the intersection of w with the hyperplane F = v, and 
where Woiv) denotes the part of v}(v) lying between the hyperplanes < == 0 
and < = 1. 

Hence the condition of similarity may be written as 

(5.25) J exp 1^"" “J f(^) dv — 0 for all o > 0 

where 

(5.26) f(v) = f (1 — * • * ^n-2) dt d^i • • • #n-2 — €• 

•^wo(v) 

By the uniqueness theorem for Laplace transforms, (5.25) implies f(v) = 0 
for all t; > 0, so that the condition of similarity finally reduces to 

(5.27) f (1 - 0""’ Pi’f'i dt dfi--- #n-2 = *. 

Of ail similar regions, let us find the one which has maximum power. Obvi¬ 
ously we want to include in w(v) all points for which / < 0. In addition we want 
to choose Wo(v) such that 


(5.28) 



(1 — O’* > ^n-.2) dt #1 • • • d^n-2 


max 


where Wb{v) is that part of w(v) in which max 



< t. 


If, for some alternative 6, Wq{v) is contained in — < / < 1, then Wh{v) and 

Wq{v) coincide and hence (5.28) attains its maximum value 6 whatever the posi- 

Tfdi Tih 

tion of Wo{v) in — < ^ < 1. If on the other hand ~ is so close to 1 that 

V V 

w5 » 

~ ^ < 1 is too small to contain Wo{v), then (5.28) attains its maximum for 

Td) 

any Woiv) containing —</<!. There exists therefore a unique Wo(v) which 
maximizes (5.28) for all values of h and y, namely the region defined by 


(5.29) C(v) < t < 1 

where C is determined by (5.27). 

Since under H 2 , the statistics 7 and T are independent, C does not depend 
on t;. The test 


(5.30) t<0, >C 

which we have just shown to be uniformly most powerful, is also the likelihood 
ratio test which completes the proof of the theorem. 

We shall finally consider an example of an optimum test in connection with a 
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rectangular distribution. Let Zi, • • • , be independently and uniformly 
distributed over (a, a + 0), where 6 is positive. For testing the hypothesis 
H: a = aoy the test 

(5.31) ^0. ^0 

where Y i and Y n are the smallest and the largest of the X's respectively, is the uni- 
formly most powerful of all similar regions. 

The proof of this goes through very much like that for Hi in Theorem 5. 
Without loss of generality we take Oo = 0. Also again, to simplify the proof, 
we restrict ourselves to regions which are symmetric in the variables. We need 
the following lemma. 

Lemma. Let Xi, • • • , he independently and uniformly distributed over 
(o, a + 9). Let Yi denote the zth X in order of magnitude, and let 

(5.32) r» = F„, ( = 1, ••• ,n - 1). 

Then for a > 0 

(5.33) p(^.•••,<«) = ’^C-i h 
when 

a < tn < a + e, — — - —;;— < 4 < 1, (ife = 1, • • • , n - 1). 

• * • 4+1 

This is easily seen by applying the usual method of Jacobians. The inequali¬ 
ties describing the sample space of the T^s are equivalent to the following more 
convenient ones: 

(5.34) a<tn<a + e,j (fc =■ 1, • • • ,n - 1). 

In 

Let us denote by w{tn) the intersection of a region w with the hyperplane 
Tn = tn f and by wo{tn) that part of w{tn) contained in the cylinder 0 < 4 < 1| 
(fc = 1, • * • , n — 1); then we find as a necessary and sufficient condition for 
w to be similar with respect to 9 (assuming H) 

(6.35) (n - 1)! f t:zl = *. 

Of all regions satisfying (5.35) we want to find the most powerful one. Let 
us first consider alternatives a > 0. If Wa(4) denotes the common part of t0o(4) 
and the region 

(556) r < • • • <1 < 1, 

we must choose ti?«(4) such that 
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(5.37) / till • Cl? • • • <2 dtn-i • • • = max. 

From this it follows easily that against alternatives a > 0 the uniformly best 
choice for ttJo(C) is 


(5.38) 


tlt2 * • • tn-l — ^ C'{tn)y 

Vn 


and since under 


Fi 

Fn 


is independently distributed of Tn 


C\t„) docs not depend 


on tn . 

Consider next alternatives a < 0. We include in the region of rejection all 
points for which Yi < 0. To determine Wq{1„) we notice that, given Fi > 0, 
the X*s are uniformly distributed between 0 and o + (Provided a + d > 
the case a + B < 0 is trivial). Hence the probability distribution of the P’s 
given Fi > 0 is 


(5.39) 


Pil'ly * * ’ > I Fl ^ 0) 


n! „-i 
(a + BY " 




when 


< tn < a + By 0 < 4 < 1 for A* = 1, • • • , n - 1. 

Thus 

/K yitl , - * , tn-l I C , Cl < 0, Fi > 0) 

p(^l > • • • > ^-1 Mn , o = 0) 

is independent of , • • • , and hence the power of w against alternatives 
a < 0 is independent of the choice of i4?o(4). Therefore the region 

(5.41) yi < 0, > C' 

Vn 

is uniformly most powerful against all alternatives. But (5.41) is equivalent to 

(5.42) —^ < 0, > C. 

Vn - yi 

It is interesting to compare this result with that for the corresponding simple 
hypothesis. Let be the hypothesis; a = 0 when the are assumed inde¬ 
pendently and uniformly distributed over (a, a + 1). There exists no uniformly 
most powerful test of H'; instead the two uniformly most powerful one-sided 
tests exist. By analogy with the normal case one might then expect for H' 
that of all tests with S3anmetric power-functions, there be a uniformly most 
powerful one. This however is not so: there exist infinitely many admissible 
tests with symmetric powerfunction. 
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In this and the previous section we restricted ourselves to problems involving 
only one nuisance parameter. However, the method applies also to problems 
involving several nuisance parameters. 

In the usual way (cf. [20, 9]) the results of this section may be translated to 
give optimum sets of confidence intervals for estimating the parameters in ques¬ 
tion. In this connection it is an open question whether the confidence regions 
based on the type Bi tests discussed in section 2 will always be intervals; one 
would expect this to be the case. 

The author wishes to acknowledge his indebtedness to Professor P. L. Hsu 
for many helpful suggestions. 

Added in proof: In a joint paper ])y Professor Henry Scheff6 and the present 
author which has been submitted to the Proceedings of the National Academy of 
Sciences, a result is given concerning the existence of certain 1; 1 transformations. 
This result bears on Section 4 of the present paper where a question arises con¬ 
cerning the existence of a 1:1 transformation. The existence of such a trans¬ 
formation is now assured and, as a consequence, the last paragraph of Section 4 
has become superfluoiis. 
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A CORNER TEST FOR ASSOCIATION 
By Paul S. Olmstead and John W. Tukey 
Bell Telephone Laboratories and Princeton University 

1. Summary. This paper proposes a new test (the ‘^quadrant sum’O for 
the association of two continuous variables. Its notable properties are: 

(1) Special weight is given to extreme values of the variables. 

(2) Computation is very easy. 

(3) The test is non-parametric. 

Significance levels (for the quadrant sum) are given to the accuracy needed for 
practical use. To this accuracy they are independent of sample size (see Fig. 1). 
The generating function of the quadrant sum is given for the null hypothesis 
(no association = independence). A limiting distribution is deduced and com¬ 
pared with the cases 2n = 4, 0, 8, 10, and 14. Extension to higher dimensions 
and application to serial correlation are discussed. 

2. Description of test (even number in sample). We shall describe the 
test as though a scatter diagram had already been drawn. The possibilities of 
direct computation from tabular data are indicated by the examples in sections 
8 and 9. 

In the scatter diagram, draw the two lines, x = Xm ^ y = Vm > where Xm is the 
median of the x-values without regard to the values of y, and ym is the median 
of the t/-values without regard to the values of x. Think of the four quadrants 
or comers thus formed as being labelled + in order, so that the upper 

right and lower left quadrants are positive. Beginning at the right hand side 
of the diagram, count in (in order of abscissae) along the observations until 
forced to cross the horizontal median. Write down the number of observations 
met before this crossing, attaching the sign + if they lay in the + quadrant, 
and the sign — if they lay in the — quadrant. Repeat this process moving 
up from below, moving to the right from the left, and moving down from above. 
The quadrant sum is the algebraic sum of the four terms thus written down. 
This process is illustrated in Fig. 2, where the black dots represent contributions 
to the sum, and the dotted lines, crossings. 

When there are an even number of pairs (x, y) and no ties, the medians will 
pass between the points. In this, the simplest case, the distribution of the 
quadrant sum is known for the hypothesis of no association (that is, of inde¬ 
pendence), and significance levels are given in Table 1 for the magnitude (abso¬ 
lute value) of the sum. It will be noticed that the sample size does not enter in 
any important way. 

The cases of an odd number of observations and of ties are discussed in the 
next two sections. Simple devices make the test usable in most cases. A very 
great tendency toward ties, however, will make it inapplicable: This will be 

495 



PAUL S. OLMSTEAD AND JOHN W. TUKEY 






8 

M 


AllllSVeOHd 


ABSOLUTE VALUE OF QUADRANT SUM 









CORNER TEST FOR ASSOCIATION 


497 


unimportant in most applications because of the fact that attention is being 
directed to the periphery. 


INDIVIDUAL TERMS 
TOP « +3 
RIGHT »+l 
BOTTOM <-|-6 
LEFT s+6‘/2 


QUADRANT SUM 

|S 1 » 16 '/2 
P & 0.8% 



Fig. 2. Scatter diagram of 116 pairs of observations 


The set of data which prompted the development of the test is shown in Fig. 2. 
The accompanying report described it as follows: ‘The various points appear 
to be scattered almost completely at random and give little indication of corre¬ 
lation,” The quadrant sum is which is significant at the 0.5% point. 
Intuitively, the significant association of the peripheral points is clear. 
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3. Description of test (odd number in sample). If the sample size is odd, 
then we may usually follow the process outlined above. We will have difficulty 
only when the counting process meets a point, one of whose coordinates is a 
median. In this case we employ a simple device, namely: 

Given a sample of 2w + 1 pairs, let rr* and y* be the medians of the a:-values 
and of the ^/-values, respectively. Let the pairs in which they occur be (x*, yk) 
and {xm , y*), respectively. Replace these two pairs by the single pair {Xm , yk)- 
There are now 2n pairs and the regular method can be applied. 

The quadrant sum so obtained from an unassociated population has the same 
distribution as that formed directly from 2n pairs. 

4. Description of test (treatment of ties). The behavior of the test is known 
when (1) there is no association, (2) the probability of a tie in x-values or y-values 

TABLE 1 


Working significance levels for magnitudes of quadrant sums 


Significance level (Conservative) 

Magnitude of quadrant sum* 

10 % 

9 

5% 

11 

2 % 

13 

1 % 

14-15 

0.5% 

15-17 

0 .2% 

17-19 

0 .1% 

18-21 


* The smaller magnitude applies for large sample size, the larger magnitude 
for small sample size. Magnitudes equal to or greater than twice the sample 
size less six should not be used. 


is zero. The following approximation, which has an unknown effect on the 
distribution, is suggested when ties are present: 

When a tied group is reached, count the number in the tied group favorable 
to continuing and the number unfavorable. Treat the tied group as if the 
number of its points preceding the crossing of the median were 

number favorable 
1 + number unfavorable’ 

It seems likely that this approximation is conservative. 

{H. Discussion. When a moderate number, say 25 to 200, of paired observa¬ 
tions en two quantities are plotted as a scatter diagram, visual examination 
frequently detects what seems to be definite evidence of association between 
the variables. Often in such cases, the usual methods for measuring associa- 
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tion do not find statistical significance of association. Visual judgment, par¬ 
ticularly by engineers or scientists who may wish to take action on the basis 
of their findings, gives greater weight to observations near the periphery of the 
scatter diagram. This is not always desirable—^but often it is very desirable. 
A quantitative test of association with such concentration on the periphery has 
been lacking. The quadrant sum test was developed to fill the gap. Its fea¬ 
tures of speed and non-parametricity are useful but secondary from this point 
of view. 

When uniform attention to the whole scatter diagram is desired, the quadrant 
sum test is of unknown usefulness. We know little enough of the operating 
characteristics of the more conventional tests, such as: 

1. The product moment correlation coefficient 

2. The four-fold table formed by the medians 

3. The biserial correlation coeflScient 

4. The rank correlation coefficient 

and less about the operating characteristics of the present test. In this case, 
the quadrant sum test can only be recommended definitely for exploratory 
investigations of large amounts of data. 

There are many situations, however, where we do not know where to concen¬ 
trate our attention, and where speed and non-parametricity are cardinal virtues 
in a test. One example is the use of serial correlation in studjdng industrial 
processes. We may guess that here we are interested in the periphery, but 
neither theory nor experience can, so far, prove this. In such situations the 
quadrant sum is by far the fastest to use of any of the tests known to the authors, 
and we believe one of the most useful. 

6. Elementaiy derivations. We can easily find the distribution of 

1. An individual term of the quadrant sum 

a. For fixed sample size 

b. In the limit 

2. The quadrant sum itself 

a. For fixed sample size 

b. In the limit, assuming asymptotic independence of the four terms. 

This we shall do now, leaving the proof that 2a actually converges to 2b to a 
later section. 

Consider a sample of 2n pairs (xi, yi), • • • , (x 2 n, yin) from a population in 
which X and y are independent. It is both clear and easily verifiable that 

1. The set of 2n x-values, Xi^ ••• ,x%n 

2. The set of 2n y-values, yi, • • • , yin 

3. The permutation of the order of the y-values when the pairs are ordered 

by the x-values 

which together determine the sample, are independently distributed, and that 
any permutation is as likely as eveiy other. (We have assumed no ties, which 
is a consequence, with probability one, of the continuous cumulative distribu- 
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tions of X and y). Since the quadrant sum depends only on the permutation, 
its distribution in the absence of association does not depend on the distribu¬ 
tions of X and y. 

We must solve, then, certain purely combinatorial problems—under the 
hypothesis that the 2n! permutations of the y-values are all equally likely. 
It may simplify matters to assume that the values of x in the sample are 1,2, • • • , 
2n and that those of y are the same. How, then, do we calculate the distribu¬ 
tion of a single term of the quadrant sum. I^et us begin with small a:-values, 
and the pair (1, yi). If 2/i = 1, 2, • • • , n, we count '*one’’ positive, and if 
yi = n + 1, n + 2, • • • , 2/1, we count ‘‘one*’ negative. We pass on to (2, y^ 
and so on. How many permutations yield a count of exactly k positive values? 
Those in which 2 / 1 , 2 / 2 , * * * ,2/* are equal to or less than n, yu^i equal to or greater 
than n + 1, and the other (2n — ifc — l) 2 /*s are arbitrary. There are: 

n{n — 1) • • • (n — A: + l)*(n)(2n — A; — 1)1 

such permutations, the fraction of all (2n)! permutations being: 

. . n(n — 1) • • • (n — A: + l)n 

^ ^ (2n)(2n - 1) ... (2n - A* + l)(2/i - k) 

which is, then, the probability that this contribution will equal +A:, or by sym¬ 
metry, the probability that it will equal — A:, A: 0. 

For large n, this becomes merely: 

(2) pk = k 7^0. 

In order to obtain the distribution of the quadrant sum itself, we must concern 
ourselves with the lack of independence of the four terms. This is indicated 
most clearly in the case of 2n = 2, where the 21 = 2 permutations yield 
-t-1 -|-1 +1+1=4 and —1 —1 —1 —1 = —4. Here, there is complete lack 
of independence. We shall see later that there is effectively independence in 
the limit, so that it is worth while to calculate the sum of four independent 
terms with the limiting distribution (2) and find that it satisfies: 

(3) Pr(\ mdependent sum of 4 terms j > A;) =--, A; > 0. 

The details will be omitted. 

A simple device, reminiscent of Wald’s [3, 1943] establishment of the two- 
dimensional tolerance limits enables us to avoid difficulties with lack of inde¬ 
pendence and compute the exact distribution of the quadrant sum for any n. 
We decompose the permutation of the 2 /-values into the following parts, which 
together specify the permutation: 

(a) The number, j, of pairs in the upper right quadrant.^ 

(b) The set of j values of x between n + 1 and 2n corresponding to pairs in 
the upper right quadrant. 
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(c) The set of j values of y between w + 1 and 2n corresponding to points 
in the upper right quadrant. 

(d) The set of j values of x between 1 and n corresponding to pairs in the 
lower left quadrant. (Note that the use of medians ensures that the 
lower left and upper right quadrants contain the same number of points.) 

(e) The set of j values of y between 1 and n corresponding to pairs in the 
lower left quadrant. 

(f) The permutation of j objects defined by the pairs in the upper right 
quadrant. 

(g) The permutation of n — j objects defined by the pairs in the upper left 
quadrant. 

(h) and (i) the permutations from the remaining quadrants. 

It is easily verified that: (1) given items (b) to (i) can be assigned at will, (2) 
each assignment of (a) to (i) corresponds to one and only one permutation, (3) the 
quadrant sum depends only on items (b) to (e). In fact, the right hand term 
depends on item (b), the upper term on item (c), the left hand term on item (d) 
and the lower term on item (e). While j remains fixed, the terms behave 
independently. 

For fixed what is the distribution of a single term? If a set of j x-values 
gives the term +ky it must contain the k largest a:-values and not contain the 
next. There are: 



such sets. The generating function for a single term, is, then: 


(4) 






Since the terms are independent for fixed j, and there are — j)!)* 

ways to supply the permutations forming items (f) to (i), the generating func¬ 
tion for the quadrant sum, Sn , is: 


(5) On(x)^± 

j-0 


Ulfgn ^j)lf 

(2n)I 





The exact probability of equalling or exceeding each value of Sn has been 
computed for 2n = 2, 4, 6, 8, 10, and 14. Table 2 gives these probabilities 
and Fig. 3 shows the values of 


^ + logio Pr( I quadrant sum | > m) 

5 

this particular function being chosen for its relative constancy. The maximum 
value of the quadrant sum is 4n, and for values of k less than 4n — 6, there 



TABLE 2 

Prohahility of a Sum of Absolute Value Equal to or Greater than k when a Sample 
of 2n is Drawn from an Unassociated Population 


\ 2n 

ifci 

2 

4 

6 

8 

10 

12 

14 

oo * 

0 

1.0000 

llffi 







1 

1.0000 


0.9333 

0.9036 

0.9106 


0.9115 

0.912037 

2 

1.0000 

RRnSir 

0.7556 

0.7544 

0.7567 


0.7580 

0.754630 

3 

1.0000 

0.4167 






0.599537 

4 

1.0000 

0.4167 

0.4667 

0.4619 




0.462963 

5 

0.0000 

0.3333 

0.3111 


0.3519 


0.3547 

0.346933 

6 

0.0000 

0.3333 

0.2222 

0.2619 



0.2611 

0.252025 

7 

0.0000 

0.3333 

0.1556 

0.1821 

0.1867 


0.1876 

0.177662 

8 

0.0000 

0.3333 

0.1111 

0.1258 




0.121817 

9 

0.0000 

0.0000 

0.1000 


0.0928 


0.0918 

0.081471 

10 

0.0000 

0.0000 

0.1000 

0.0554 

0.0642 



0.053295 

11 

0.0000 

0.0000 

0.1000 

0.0375 

0.0436 


0.0432 

0.034189 

12 

0.0000 

0.0000 

0.1000 




0.0296 

0.021557 

13 

0.0000 

0.0000 

0.0000 




0.0202 

0.013386 

14 

0.0000 

0.0000 

0.0000 


0.0127 


0.0139 

0.008200 

15 

0.0000 

0.0000 

0.0000 


0.0095 


0.0096 

0.004963 

16 

0.0000 

0.0000 

0.0000 


0.0083 


0.0066 

0.002972 

17 

0.0000 

0.0000 

0.0000 


0.0079 


0.0045 

0.001762 

18 

0.0000 

0,0000 

0.0000 


0.0079 


0.0031 

0.001036 

19 

0.0000 

0.0000 

0.0000 


0.0079 


0.0021 

0.000604 

20 

0,0000 

0.0000 

0.0000 


0.0079 


0.0014 

0.000350 

21 

0.0000 

0.0000 

0.0000 




0.0010 

0.000201 

22 

0.0000 

0.0000 

0.0000 




0.0008 

0.000115 

23 

0.0000 

0.0000 

0.0000 




0.0006 

0.000065 

24 

0.0000 

0.0000 

0.0000 




0.0006 

0.000036 

25 

0.0000 

0.0000 

0.0000 




0.0006 

0.000020 

26 

0.0000 

0.0000 

0.0000 




0.0006 

0.000011 

27 

0.0000 

0.0000 

0.0000 


3 ^^ 


0.0006 

0.000006 

28 

0.0000 

0.0000 

0.0000 


3 ^^ 


0.0006 

0.000003 

29 

0.0000 

0.0000 

0.0000 


3 ^^ 


0.0000 

0.000002 


0.0000 

0.0000 

0.0000 




0.0000 

0.000001 

31 or over 

0.0000 


0.0000 

0.0000 

0.0000 


0.0000 

0.000000 

Variance 









of A; 

16 

24 

26| 

26f 

26M 

26A 

26m 

24 


‘ Probability for 2n == oo, fc > 0, is given by 

m + m + 168A; + 208 
216 • 2* 
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is quite good agreement between the curves for finite n and formula (3) at 
the practically significant percentage points. The situation for very small 



4 8 12 16 20 24 28 

ABSOLUTE VALUE OF QUADRANT SUM 


Fio. 3. Comparative relationships for finite and infinite sample sizes and 
normal approximation to the infinite sample size 

probabilities suggests a careful consideration of the limiting behavior of the 
quadrant sum distribution (see section 10). 
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The device for samples of 2n + 1 deserves a word of justification. If there 
is no association, the 2n + 1 y-values are randomly paired with the 2n + I 
a:-values, and, in particular, the ^-value paired with the x-median is randomly 
selected. If we pair it with the (randomly selected) a;-value which was paired 
with the ^/-median we still have random pairing. The pairing of the 2n pairs 
is random, although neither the x-values nor the ^/-values make up a sample. 
The randomness of pairing is all that has been used in the discussions of this 
section. 

7. Extension to higher dimensions. The same ideas that underlie the quad¬ 
rant sum test for two variables may be extended in several ways to give tests 
for various types of association among three or more variables. Only one 
three-variable case will be discussed here, leaving further extension to the 
reader. 

Given three variables, x, y, and z, and a sample of matched observations on 
these, it is clearly possible to use the simple quadrant sum test for two variables 
to investigate association between x and y separately, between y and z separately, 
and between z and x separately. If the Pearson coefficient of correlation were 
being computed and were found to be close to zero for each of these pairs, it 
would be assumed that there was no detectable association through the second 
moments. In a trivariate normal or Gaussian distribution, where the first and 
second moments determine the whole distribution, if there is independence be¬ 
tween the separate pairs of variables, there is no possibility of a three-way 
association. It is of some interest, however, to notice that a corner sum test 
can be devised that will measure the effect of such triple association in case it 
does exist. 

Consider the octants into which the three median planes for x, y, and z, 
respectively, divide the three dimensional scatter diagram and label the octants 
alternately plus and minus, in the manner suggested by Fig. 4. More precisely, 
an octant is counted as plus if an odd number, that is three or one, of the vari¬ 
ables are greater than the medians of the sample, and the remaining octants are 
labelled minus. It is clear that we may repeat the process of coming in along 
each axis passing from observation to observation as long as they remain in a 
region of fixed sign, and writing down as a contribution to the final or octant 
sum the number of such consecutive elements and the sign of the region in which 
they were found. There will be six terms rather than four, as was the case 
for the test based on quadrants, and so a new set of significance levels will be 
required. Table 3, following, lists the situation for a very large sample. 

The situation has been sketched for the case of 2n triples. If there are 2n + 1 
triples, then we may have trouble with the medians again. However, a similar 
device works, except that we must agree on a last variable in order to form the 
S 3 rnthetic triples uniquely. For example, consider the triples (m, 3, 5), (9, m, 1), 
(12, 4, m), where m denotes the median. Taking the order in which the vari¬ 
ables are written, we get (12, 3, 5) and (9, 4, 1) as the synthetic triples. Other 



Fig. 4. Octant schematic—solid sections taken as positive 


TABLE 3 

Working significance levels for the magnitudes of the octant sum 


Significance Level 

Magnitude of Octant Sum* 

10% 

11 

5% 

13 

2% 

15 

1% 

16 

0.6% 

18 

0.2% 

20 

0.1% 

21 


* Computed for large samples only and based on normal approximation, see 
section 11 for discussion of this and higher dimensional cases. 
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orders would yield (9, 3, 6) and (12, 4, 1) or (9,3,1) and (12,4,5). This slight 
dissymmetry is not pleasing but should give no difficulty. 

8. Nongraphical example. The following example of 78 successive observa¬ 
tions of four variables shows how this test may be applied without plotting and 
how simple the computation still remains. The data concern a metallurgical 


TABLE 4 

Excerpt from TippeWs Table 


Time T* 

Fuel F* 

Material M* 

Articles A* 

Duration D* 

1 - 

246 + 

1457 - 

1895 + 


2 - 

196 - 

2078 + 

2121 + 


3 - 

192 - 

1278 - 

1437 - 


4 - 

202 + 

1398 - 

1497 - 


5 - 

206 + 

1944 + 

1592 + 


6 - 

218 + 

1464 - 

1506 - 


7 - 

155 - 

1541 + 

1762 + 


8 - 

201 + 

1502 + 

1818 + 


9 - 

211 + 

1950 + 

1144 - 


10 - 

236 + 

1768 + 

1654 + 


etc. to 





‘ 78 + 

185 - 

1536 + 

1442 - 


Median 

Median 

Median 

Median 

Median 

39.5 

199 

1474 

1588 

149.5 


* Location of observation relative to column median; + = above; — = below. 

Tippett’s correlations (based on lightly rounded data) 

Tfm = + 0.243 

Tfa = + 0.266 
Tma = + 0.681 
Tfm,A = "h 0.088 
riPMA, = + 0.141. 

problem in mass production and are taken from L. H. C. Tippett, Table XXII, 
page 63 [2]. An excerpt from the data is given in Table 4 together with Tip¬ 
pett’s calculated correlations. This table also shows the preliminary marking 
of each individual measurement as above (+) for its variable, below (—), or 
on the median (0). From this table we see, for example, that increasing T con¬ 
tributes a term —3 to the quadrant sum for T and D, It is often desirable to 
prepare auxiliary tables to assist in computing the components of the quadrant 
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and hyperquadrant sums. Such a table is Table 5 for low values of Fuel (F—) 
arranged in consecutive ascending numerical order. The entries on this table 
for the five columns headed F, T, ilf, A, and D are directly comparable to the 
entries in Table 4. For example, F = 155 is ~ with respect to the fuel median 
and T = 7, —; ilf = 1541, +; A = 1762, +; Z) = 152, +. The double, triple, 
quadruple and quintuple headed columns contain simply the algebraic multi¬ 
plication of the signs in the appropriate T, M, A, or D columns. Thus, TM 
for F = 155 is —, MAD is +, and TMAD is —. The contribution to each 
quadrant or hyperquadrant sum is simply the count of the consecutive like 
signs from the top of a column. For column AD, we have 7 consecutive + 
signs and since the contribution is to FAD and F is —, the contribution in this 
case to the octant sum is —7. The results from the ten tables of which Table 5 

TABLE 6 


Sample Table for One Component of Quadrant and Hyperquadrant Sums. Low 
Values of Fuel (F—) 


FuelF 

T 

Af 


D 

TM 

TA 

TD 

MA 

MD 

AD 

TMA 

TMD 

TAD 

MAD 

TMAD 

98 - 

+ 

— 


— 

— 

— 

— 

4- 

4- 

4 - 

4 

4 

4 

— 

— 

135 - 

+ 

— 

— 

— 

— 

— 

— 

4- 

4- 

+ 

4 

4 

4 

— 


140 ~ 

— 

— 


— 

4- 

4* 

4- 

4- 

4- 

+ 

— 

~ 

— 

— 

4 

146 - 

— 

— 

— 

— 

4- 

4 * 

4- 

4- 

+ 

4- 

— 

— 


— 

4 

147 - 

+ 

+ 

- 

- 

+ 

- 

— 

— 


4 



4 

4 

4 

149 - 

— 

+ 

— 

— 

— 

4- 

+ 

— 1 

— 

4 

4 

4 

— 

4 

— 

161 - 

+ 

— 

— 

— 

— 

— 

— 

4- 

4- 

4 

4 

4 

4 

— 

— 

163 - 

+ 

- 


— 

— 

+ 

— 

— 

4 - 

— 

— 

4 

— 

4 

4 

165 ~ 

— 

+ 

+ 


— 

— 

— 

4- 

4- 

4 

— 

— 

— 

4 1 

- 


Contributions to Sums 

FT FM FA FD FTM FT A FTD FMA FMD FAD FTMA FTMD FT AD FMAD FTMAD 

-2 +4 +7 +8 +2 +2 +2 -4 -4 -7 -2 -2 -2 +4 +2 


is a sample are then carried to the summary computation shown in Table 6. 
The contribution from Table 5 is shown on line F—. The totals are computed 
and their probabilities of occurrence determined. 

9. Serial example. The following example, a sample of 144 observations of 
the thickness of inlay for relay springs cut consecutively from a single sheet of 
material, allows us to compare the resolution of the present test with that of 
the serial product-moment correlation. The data are from Shewhart [1, 1941, 
Table 1] and the serial correlations from lag 1 to lag 22 are from recent calcu¬ 
lations by Miss Dorothy T. Angell. The procedure for calculating the serial 
quadrant sums is similar to that for obtaining the sums for section 8. A table 
is prepared to show the observed consecutive order of the numerical values and 
each is identified as above (+), below (—), or on the median (0). This gives a 






TABLE 6 

Summary Computation Table for Quadrant and Hyperquadrant Sums 


COC^ F-IC^ f-HCO c«co 

avHdx ++ +4- ++ +4- ++ 


^ ^ eo eo 

I I 

_ _ »—I»—I 

I 



Significant at 5%.... 
Significant at 1%.... 
Significant at 0.2%.. 
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table similar to one of the elements, say Fuel, in Table 4. Four computation 
tables similar to Table 5 are required, one for the equivalent of moving from 
the right, one from below, one from the left, and one from the top of a lag cor¬ 
relation scatter diagram. One table from each direction will take care of all 
lags. In the first, the marginal entries are the observed values listed in descend¬ 
ing numerical order. Opposite these are recorded from the previous table the 
signs associated with observations for each lag with respect to each entry. 
The second table would record the signs relating to the lags from the observed 
values arranged in ascending order. The third table would record the signs 
relating to leads from the observed values arranged in ascending order and the 
fourth, the signs relating to leads from the observed values arranged in descend¬ 
ing order. The sign of the contribution from each group is the algebraic product 
of the sign of the run and the sign of the marginal entries. The length of run 
is determined in the same way as in Table 5. Table 7 illustrates the procedure 


TABLE 7 

Rdaiion of Lagged Observations to Median (4- above, — * below) for Smallest Observation* in Ascending Order 

25 

+ 

(+) 
+ 

+l 

* Contribution to Serial Quadrant Sum. 


Thick- 

Lag 

neu 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

n 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

2 

_ 

- 

+ 


- 

+ 

- 

+ 

- 

4- 

- 

4- 

4- 

4* 

4- 

+ 

4- 

4" 

4- 

4- 

4- 

4- 

4- 

4- 

— 

8 


- 

+ 

- 

+ 

- 

- 

- 

- 

4- 

- 

- 

4- 

- 

4* 

- 

4- 

- 

4- 

4* 

+ 

4- 

+ 

4- 

4- 

8 


— 

— 

+ 

- 

- 

— 

— 

+ 

+ 

— 

4- 

4- 

— 

4* 

4- 

— 

— 

4- 

4- 

— 

— 

— 

— 

— 

10 

- 

+ 

— 

— 


- 

+ 


- 

4- 

4- 

B 

4- 

4* 

- 

- 

4- 

4- 

- 

- 

- 

- 

— 

- 

4- 

13 

_ 

- 

- 

- 

- 

+ 

+ 

•f 

- 

4- 

+ 

4- 

4- 

(4-) 

(4*) 

(4-) 

(+) 

(4-) 

(4-) 

(4-) 

(4-) 

(4-) 

(4-) 

(4-) 

(4-) 

17 

— 

— 

— 

+ 

— 


+1 

— 

+ 

— 

4- 

— 

4- 

4* 

4- 

4- 

4- 

+ 

4- 

4- 

4- 

4- 

4- 

4- 

4* 

17 

— 

+ 

- 

+: 

+ 

+ 

+ 1 

+ 

+ 

- 

- 

+ 

4* 

- 

- 

- 

_ 

- 

- 

+ 

- 

D 

4- 

- 

4- 

18 

- 




+ 

+ 

“1 

+ 

4- 

- 

4- 

4- 

— 

— 

4- 

4- 

_ 

— 

— 

— 


B 

+ 

— 

4* 

• 


+3 

-2 

4 

+1 

-1 

+3 

-1 

+2 

-6 


-1 

2 

-1 

-3 


-2 

-1 

-3 

-8 

-2 

-2 

-2 

-2 

4-1 


of determining the contribution from lags associated with the observations 
arranged in ascending order. 

Two serial quadrant sums may be computed—a circular serial quadrant sum 
or a noncircular serial quadrant sum. Circular items arise from considering 
that the beginning of the set of observations is a continuation of the end in the 
same way that this assumption is made in computing circular serial correlation 
coefficients. In Table 7, circular items are shown in parentheses and are omitted 
in calculating noncircular sums. In the particular table shown, the count of 
the run lengths was identical for both types of sum, but in other cases this may 
not be the case. Since the serial quadrant sum is relatively insensitive to 
sample size, the noncircular serial quadrant sum has for all practical purposes 
the same distribution as the circular quadrant sum. The correspondence in 
thia case between the serial correlation coefficient for each lag up to 22 and 
the respective values of the two types of serial quadrant sums is shown in Fig. 6. 
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Fig. 6. Comparative perfonnance on a serial (autocorrelative) example 

10. Convergence to the ^ting distribution. We shall consider several 
chance sums. One of these is S, which has the limiting distribution discussed 
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-(e 

\i-l 


2“"+"x* + E 2- 

t-1 


X 


■) 


The total probability assigned to iSf* = — jfc — — i') ». • , 

so that there is nonzero probability that k L not Lnid ’ ?irtSiST' 

generating function is (5), and the fourth is the 
result of the same sort of curtailment applied to S h will k *i? 

-S,.* and its generating function is " ‘ * ^ 

This again corresponds to a total probabUity less than unity 
It IS clear that J| ‘ 


and 


Pr(S,.t = m) :< Pr(S„ = m) 
Pr(Si = m) ^ Ft(S « m). 


We shall soon show that 

^r(Sn.i, = m) = Pr(S* = w) 

and this will imply that 

lim Pr(Sn = m) = Pr(<S = m) 

yjich is the desired result. The implication runs as follows: given . we can 
choose k so laige that 

Pr(<S* defined] > 1 — f/3 

whence 


I Pr(<Si = m) — Pr(iS = m) | < «/3 
and then choose n so laige that 
I Pr('S».» = m) — Pr(/S* «= »») | ^ */(241: + 6) 

forTO= -4A,-44+1,-..,4* 

whence 


Pr ( iS«4 defined ) > 1 - e /3 — ^ . <• i _ 164 + 3 

244 + 6*^^ 2i4 + 6* 
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and hence 

I Pr(5',,» = m) - Pr(,Sf, = m)\ < « 

this inequality holding automatically for | m | > 4k. Hence, 

I Pr(^r, = m) - Pr(^ = m) | 

< I Pr(5» = ot) - Pr(S„.t = m) 1 + I Pr(«„,* = w) - Pr(5i = m) | 
+1 Prffi; .- P,(« . ») I < . + 5iyi,-5 . + i. < . 


This method is clearly of general application in such problems. 

We turn now to the proof of (6). The expression for GnA^) shows that we 
may consider it the result of the following process: the integer j is a chance 
quantit 3 ^ with the distribution 


Pr(i = jo) 


(njf 

( 2 / 1 )! 


For fixed jy Gn,k is the average over j of 




The first of these relations shows that j/n converges stochastically to J as n 
approaches infinity. The second shows, since 

/n - i - 1 \ 

\n - i - 1/ ^ (n - i - l)!(?e, - j)[jl ^ {n - j ) {J)U - 1) - ■ (j - ^ + 1) 

(n — j — 1)!(; — i)!^! n{n — l)(n — 2) ... (n — t) 

i “ = (n i — l)!(n — j)ljl 

(n — j — t)l(j — l)!n! 

^ (n - j)(n - i ~ 1 ) « - (n - i - ^ + l)j 
n(n — 1) . •. (n — i) 

and both of these converge stochastically to 2“^*^^^ as n approaches infinity, 
that Gn,k,i{x) converges stochastically to Gk{x). Since these curtailed generat¬ 
ing functions involve only powers of x in the finite range between — 4A: and +4A;, 
the limiting relation (6) follows at once. 




11. Effectiveness of normal approximation. Fig. 3 shows the relation be¬ 
tween the asymptotic distribution of the quadrant sum for large n and a normal 
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distribution with variance 24, i.e., the same variance as that of the asymptotic 
distribution. The normal approximation is calculated from 

Pr(|S«| 

where x is normally distributed with zero mean and unit variance. The as 3 nnp- 
totic and normal curves agree surprisingly well out to the 5% point, and an error 
of a full unit in the significance level first occurs beyond the 0.5% point. 

Since the asymptotic distributions for the quadrant, octant, hexadecant, do- 
triacontant,—, sums become more and more normal, the normal approximation 
will be even better for higher dimensions. In r dimensions, this approximation 
consists in treating 

\Sn\- h 
Vl2r 

as the absolute value of a standard deviate. This should be quite adequate for 
large samples and r > 4. 

12. Unsolved problems. The central unsolved problem in connection with 
the quadrant sum is: 

(1) What is the operating characteristic? 

This has as a corollary the more general question: 

(2) I low can the operating characteristic of a nonparametric test be de¬ 
scribed so as to be useful to the users of the test? 

There are, of course, minor problems which are much more easily soluble. A 
few, listed in order of practical importance, are: 

(3) What is the effect on the significance levels of the use of lagged values 
of a; as values of y? 

(4) What are the exact distributions for moderate n in three or more dimen¬ 
sions? 

(5) Do the analogous limiting distributions hold for three or more dimen¬ 
sions? 

(6) What is a better approximation to the limiting distribution for moderate 
n? 

To encourage others to solve some of these, we close with the assurance that 
they have our good wishes. 
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DISCRIMINANT FUNCTIONS 

By George W. Brown 
Iowa State College 

1. Introduction: In the following sections the development of discriminant 
function techniques is approached from an elementary point of view, considering 
first an essentially trivial problem, then working up to the more complex situa¬ 
tions which may be handled by discriminant function methods. No attempt 
has been made to follow the pattern of the historical development in this process, 
and no consistent attempt has been made to allocate proper credit, in the text, 
to those individuals responsible for the introduction and exploitation of these 
methods. A more or less exhaustive bibliography of discriminant function 
applications and related theory is given at the end of this paper. 

Some historical perspective may be gained, however, from a very sketchy 
consideration of the early background of the subject. The first published 
application of the discriminant function seems to have been the work of Barnard 
(1935 [1]) on craniometry, following the suggestion of R. A. Fisher. Meanwhile 
P. C. Mahalanobis (1927, [30]; 1930, [31]) and, in this country. Hotelling (1931, 
[25]) had been concerned with a closely related problem, the construction of 
measures of the “distance” between two sets of multiple measurements, for which 
Karl Pearson’s (1926, [34]) coeflScient of racial likeness was not wholly adequate. 
Fisher (1936, [18]) gave a further example of the method and showed (1938, [19]) 
the relation between his work and that of Hotelling (1931, [25]; 1936, [27]). Thus 
the theory of discriminant function analysis proper is about ten years old, but is 
intimately related to researches which go back a few more years. 

A simple problem: Consider the very simple case of a single measurement, say 
f, which may be made in each of two populations, and let us suppose, for the 
sake of discussion, that { is normally distributed, with unit variance, in each 
population, but with possibly different means in the two populations. 

Let 


Eiii) = a ~ i8 
jE2(f) = a + ^ 

be the mean values of f over the two populations, with jS > 0. As an example, 
we may consider the pH measurements of Iowa soil samples (Cox and Martin, 
[12]), for two soil populations, distinguished by the presence or absence of Azoto- 
bacter. From 100 samples, containing Azotobacter and 186 samples containing 
no Azotobacter, we have the estimated averages of pH equal to 7.423 and 6.016 
respectively, with an estimated standard error of .625 within populations (see 
Rg. 1). 
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OL = 6.719 
)§== .704 
.625 
= 1.13. 

Let us suppose further that { is the only measurement available on a single 
individual, not knowing to which of populations 1 and 2 the individual belongs. 


distribution of pH Measurements 



GO! 5 €1713 7423 

Fig. 1 


The problem is to classify this individual as a member of population 1 or popula¬ 
tion 2. It is clear that € furnishes the only information on which to base a 
decision, and that essentially the only procedure available is to choose a number, 
say & , such that we choose population 1 when f < fo and population 2 when 
€ > fo • Furthermore, it is evident that the expected accuracy of classification 
depends on the size of If we wish to have equal risks of misclassification for 
members of the two populations we choose & == a. Then the probability of 
misclassification is given by P{€ > iS}, where € is a normal deviate with unit 
variance. As one would expect, the probability of misclassification tends to 0 as 
jS —► 00 and tends to i as j5 —> 0. In the Azotobacter example, if we assume 
that the estimates given are the population values, we choose (o ~ 6.719. The 
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ratio ^l<T — 1.13 is exceeded approximately 13% of the time in sampling from 
the normal distribution, leading to .13 as the probability of misclassification. 

Consider now the slightly more general situation in which we consider a fixed 
variate, say w with measurements f distributed, for fixed with a mean of the 
form a + This is the standard regression situation. As before assume 
that f is normally distributed about this mean with unit variance, that is 

f = O' 4“ + € 

where a and are constants, w may take on any or all real values, and € is a 
normal deviate. Note that if w is restricted to take on only two values the 
structure reduces to the first structure considered. An example of the continu¬ 
ous type might be constructed by considering w as genotypic yield of grain and f 
a phenotypic measure of yield (Smith, [36]). 

The simple problem formulated for the two-population case may be reformu¬ 
lated here as follows: Given the relationship f = a + /Sty + c, and given f for 
an individual for which no other information is known, how shall we estimate ty? 
For selective breeding the problem may be to select individuals for which ty is 
at one end of the scale, rather than to estimate w itself. Whatever decision is 
to be made, it is still clear that f furnishes the only available information, and 
that the certainty of the decision is a function of /3. Since (f — a)//3 = ty + c/jS, 
the variance of this estimate of ty is l//3^. Note that confidence intervals for ty, 
given f, may be constructed from the normally distributed quantity { — a — /Sty. 

It should be pointed out that in the usual regression case we are interested in 
predicting f for given ty, with the hypothesis as stated above, whereas in this 
case J will be observed, and the problem is that of estimating, as a parameter of 
the distribution of f, the fixed variate ty. 

Obviously /S must not .vanish if { is to perform any discrimination among w 
values. In practice, of course, a and /S will not be given as known values and the 
variance of t will not be known, but a finite set of observations may be available, 
for which ty values are known and f has been observed. The usual analysis of 
variance provides a significance test for the non-vanishing of /S, which is equiv¬ 
alent to testing for the significance of the regression of { on ty. 

It is to be noted that this analysis reduces to the conventional between-within 
analysis (F or ^-test) when we have the special case of two populations. More¬ 
over, if we had treated f as the fixed variate instead of ty, and considered the re¬ 
gression of ty on 5, the Analysis of Variance would have differed only in replacing 
2(f — J)* throughout by S(ty — tZ))^ and the relevant F-test would have been un¬ 
changed. 

When probabilities of misclassification are estimated from finite samples, as 
in the soil classification example, there are three sources of error, sampling error 
in the estimate of the separation value fo , sampling error in the estimate of the 
distance between the population means, and sampling error in the estimated 
standard deviation of f within populations. It does not appear difficult to set 
up confidence intervals for the probability of misclassification, assuming repeated 
classification of individuals given fixed initial samples. 
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2. The one-dimensional discriminant function. We have been dealing so far 
with the simple situation in which only one measurement per individual is 
available for purposes of discrin ination. Suppose we still have this measure¬ 
ment, call it fi, now, but we have other measurements as well, say { 2 , • • • , fp . 
As before . For the moment suppose that the remaining 

measurements have mean values independent of w, so that 

, (m = 2, • • • , p), 

and let us assume also that the {€m} are mutually independent, (m = 1,2, • • • , p) 
and are normal deviates with unit variance. It is safe to assume that nobody 
would ever argue, in this case, that the measurements { 2 , * * • , {p , provide 
information about the w value for an individual. If, then, we were so fortunate 
that we were in this situation, and knew so, we could say that is our dis¬ 
criminant function, since, if any discriminating is to be done, {1 has to do it. 


TABLE 1 


Analysis of Variance for Regression 



d.f. 

Sums of Squares 

Regression 

1 

r’Stt - |)‘ 

Error 

N -2 

(1 - r»)S({ - J)‘ 

Total 

AT - 1 

S(f - |)» 


2(f - D{w - g)) 


V2({ - - wy 


Suppose, now that the measurements , f 2 , • • • , Jp are not explicitly avail¬ 
able, but that we are able to observe a linearly equivalent set Xi, a?*, • • * , Xp, 
related to the {fm} by the transformation 



n->l 


where the Inn are unknown. For fixed Xm has expected value 


an + Ll^W 


= On -h hnWy 


SO that in general each Xn observation provides information about w. More¬ 
over, the Xn are not in general mutually independentit is evident that the 
population matrix of variances and covariances for fixed w is given by o-mn ** 

Imklnh • 

ik.l 

As an example of a set of correlated measurements, consider the Azotobacter 
example referred to above. In addition to pH values, determinations of avail- 
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able phosphate content and total nitrogen content were made on soil samples 
in each of the two populations. Means were as follows: 

pH Phosphate Nitrogen 

Mean of 100 samples with Azotobacter 7.423 133.120 29.400 

Mean of 186 samples without ” 6.015 51.113 21.140 

Mean difference 1.408 82.007 8.260 

Clearly the differences are proportional to the hypothetical 6m's. The variance- 
covariance matrix, estimated from the 284 degrees of freedom within populations, 
is given by Table 2. 


TABLE 2 



pH 

Phosphate 

Nitrogen 

pH 

111.0879 

2,292.7192 

198.4026 

284((r„„) = Phosphate 


1,042,799.1890 

5,066.2645 

Nitrogen 



29,422.3655 


Estimated correlation coefficients within populations are not large, .213 for pH 
and Phosphate, .110 for pH and Nitrogen, and .029 for Phosphate and Nitrogen. 

Another example is furnished by Fisher^s Iris measurements [8], provid¬ 
ing sepal length, sepal width, petal length, and petal width for each of 50 
individuals of Iris setosa and 50 individuals of Iris versicolor. This example is 
an unfortunate one in that either petal length or petal width alone is sufficient 
to discriminate the two populations as completely as anybody has a right to 
expect an 3 d/ime. The petal lengths, for example, vary between 1.0 and 1.9 cm. 
for the 50 setosa, and between 3.0 and 5.1 cm. for the 50 versicolor. 

Let us proceed, under the assumption that available measurements, Xm , 
are distributed normally about mean values Om + hmW, with variance covari¬ 
ance matrix amn for fixed w, keeping in mind the underlying model of fi, & , • • • , 
fp , with 

Xm = i fmn in f fl = «! + + €i ; = 012 + €2 ; • * • ; fp = ttp + €p . 

n.-l 

The skeptic may wish to grant the first part of our assumptions without grant¬ 
ing the hypothetical structure of f’s underlying the x’s. Hotelling’s work [27] 
shows that such an underlying structure of f’s may always be provided, given 
the distribution of x*8 for fixed w. In other words, a distribution of x’s for fixed 
w leads essentially uniquely to an underlying f model. 

The discriminant function, given <rmn , Um and 6m , for w, n, = 1, 2, • • • , p, 
is 
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= 'tl = '^ tni 


hiiIImI 


where 


= f: <r%„. 


and 


is the reciprocal matrix to (r«„ . That is cr*"" are the solutions of the linear sys¬ 
tems [17j 

0 ^*<r»n = 0 if m — n; m, n, = 1, 2, • • •, p 


Z-f = 1; m = 1, • • •, p. 

«—1 

That X, as defined above, is properly called the discriminant function will be¬ 
come evident immediately. Putting bm = Zmij3, x, = ]£) , we have 


Z = /9 £ 

m,n,k, 

9 

Recalling that the v"" are reciprocal to <r„, = £ l,nklnh, it can be seen that 

k 

]C = 1 if & = 1, and vanishes for k =f= 1, It follows that 

mn 

X = Pi , 

in other words, X calculated as 23 from known population quantities 

mn 

is proportional to the hypothetical , the only one of the underlying measure¬ 
ments which is related to w, thus justifying the term discriminant function for 
X. It is clear that any other linear function of the x*s is also a linear function of 
the f's, and can discriminate, at best, only as well as X itself, since all the {'s 
are independent of w, with the exception of fi. X itself discriminates w to the 
same extent that , were it available, would discriminate. 

The degree of discrimination of w*q depends, as indicated in the previous sec¬ 
tion, on the ratio of the mean square of {i, among w'a (mean square for regres¬ 
sion), to the mean square of {i for fixed w (mean square for error). Since X 
is proportional to , the same is true when X is substituted for fi. It turns 
out, of course, that X is that linear combination of x*a for which the ratio of the 
mean square for regression to the mean square for error is a maximum, or, what 
is the same thing, X is that linear combination of X’s which has the maximum 
correlation with w. From any point of view X appears to be the logical function 
of x*8 to compute. It is clear that XX is precisely as good as X, if X is any con¬ 
stant. 
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In the two population case, where w takes on only two values, X is evidently 
proportional to , where /Xmi and /Xw 2 are the mean values of 

Xm in the two populations. X is here the particular linear combination of 
for which the ratio of the mean square between populations to the mean square 
within populations S*a maximum. The value of this ratio, which measures the 
degree of discrimination possible, depends on the spread of the means ofX 
between the populations, or in general, on the spread of the means of X over some 
given distribution of ty’s. Given Cmn and hm the larger the spread of w values 
the better overall discrimination will be obtainable. On the other hand, the 
coefficients for X depend only on amn and hm . 

Since X is proportional to , it follows that the discriminant function is in¬ 
variant under non-singular linear transformation of the x's, that is, if some set of 
y% linearly dependent on the x% had been observed, together with their means, 
variances and covariances, the discriminant values would not have changed. 
This invariance is obviously a desirable property, and as such was one of the 
goals of Fisher, Hotelling, and Mahalanobis. One more property of the dis¬ 
criminant function is of interest; X is essentially equivalent to the maximum 
likelihood estimate of w. 

In our statistical model w plays the role of a fixed variate or population param¬ 
eter, and the a;’s have a joint distribution about linear functions of to as means. 
Suppose now that (o-mn) and [hm] are estimated from an analysis of variance 
and covariance on data for which w as well as x values are known. The problem 
of estimating w for a single individual whose x measurements are given resolves 
into a two-stage estimation process, the first stage being the estimation of 
(<rmn) and {6ml from the initial data, the second stage being the estimation of w 
by the discriminant function whose coefficients are computed from the es¬ 
timated {(Tmn) and {6m}. It has already been pointed out that X is the linear 
combination of a:’s which has greatest correlation with w. It turns out, then, 
that the coefficients of X are proportional to those wliich would have been ob¬ 
tained from a formal regression analysis of ty on Xi, a; 2 , • • * yXp , considering the 
x’s as independent variables and w as dependent variable, a direct interchange of 
roles as compared with the statistical model we have assumed. Of course two 
linear functions differing only by a factor of proportionality are equivalent in 
discrimination. If the formal analysis of variance is carried out for testing the 
significance of the regression of ty on a;i, 0 : 2 , • • • , , the relevant F ratio re¬ 

mains a valid test for the non-vanishing of the 6m in spite of the inversion of 
dependent and independent variables. The analysis of variance is given in 
Table 3. 

R is, of course, the conventional multiple correlation coefficient. An equiva¬ 
lent analysis can be carried out for X itself, allowing sufficient degrees of freedom 
for the estimation of the constants in X, as given in Table 4. 

This analysis is proportional to the analysis given above. It might be noted 
that the mean square corresponding to error sum of squares in this analysis is 
S(r**"6m6n, which is X evaluated for arn = 6n, (n = 1,2, • • • ,p). 
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In the Azotobacter example, Cox and Martin arrive at a discriminant function 
which has the analysis given in Table 5. 

It is evident that the difference between populations is highly significant. The 
choice of scale for X in this case forces the sum of squares within populations to 
be equal to the difference between the mean X values for the two populations. 
Thus the mean X differs by .021777 for the two populations, and has an esti- 

TABLE 3 


Analysis of Variance for Regression 



d.f. 

Sums of Squares 

Regression 

P 

R^l(w — ©)* 

Error 

N — p — 1 

(1 ~ R^)'z{w — wy 

Total 

N - 1 

2 (to — 


TABLE 4 


Analysis of Variance for X on w 



d.f. 

Sums of Squares 

Regression 

V 

fl»2(x - 


Error 

AT - p - 1 

(1 - B*)2(X - X)* 


Total 

N - 1 

2(x - xy 


TABLE 5 


Analysis of Variance of Discriminant Function 



d.f. 

Sums of Squares 

Mean Square 

Between populations 

3 

.030842 

.01028 

Within populations 

282 

.021777 

.00007722 

Total 

285 




mated standard error, within populations, equal to \/.00007722 = .008788. 
Half the difference, divided by the standard error is the normal deviate cor¬ 
responding to misclassification, if equal risks are taken. In this case the value 
of the normal deviate is 1.24, approximately, leading to an estimated probability 
of misclassification of about .11, which is not very much better than the .13 
which one would have obtained if pH alone had been used. 

In this problem, as in conventional regression analysis, it is .tempting, for 




522 


QBOROE W. BROWN 


various reasons, to consider the possibility of using smaller sets of classifying 
measurements. Moreover, a significance test for this situation is in general 
more interesting, as a practical matter, than the significance test for differences 
among populations, since the initial presumption is that we are interested in 
being able to discriminate, on the basis of Xi y X 2 ,•••, Zp , Suppose, for ex¬ 
ample, we wish to test whether the discriminant function X^p) based on Xi , 
X 2 y • • • , ajp is significantly better than the discriminant function X(^r) based on 
9 9 with r < p. The relevant test is precisely the same as the test 

TABLE 6 


Analysis of Variance for Rejecting a^r+i, • • • , Xp 




Sums of Squares 

d.f. 

Si 

Regression on 

Xly ••• yXr 

r 

-S’p 

Regression on 

Xi y • * • , Xr y Xr+1 9***9 

V 

S% - 

Difference 


p — r 

/Sr — /Sp 

Error 


A - p - 1 


Total 


N - 1 


TABLE 7 

Analysis of Variance for X = Xo 



Sums of Squares 

d.f. 

4 

Regression on Xq 

1 

4 . 

Regression on Xi, • • • , Xp 

P 

s% - 

Difference 

p - 1 

- -S* 

Error 

N - p- 1 

Sr 

Total 

N - 1 


calculated formally from the regression of w on the sets Xiy ••• y Xr and Xi, 
x%y ••• y Xpy with the analysis of variance given in Table 6. 

Similarly, if we wish to test for the significance of a theoretical discriminant 
function, Xq , with preassigned coefficients, as compared with Xp , we have 
again the conventional test calculated from the formal analysis of the regression 
ofwonXlyX 2 y•^•yXpy as given in Table 7. 

As shown by Fisher [21] the relevant F-Test for this hypothesis is computable 
as 

T? _ n — p + 1 R'^ 

1 
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where = iJ*(l - r’), r is the correlation between X and Xo for fixed w, and 
R is the multiple correlation for wonxif • • • , a?, , or, what is the same thing, the 
correlation of w and X, 

The example of Smith [36] is an example in which the relationships of x^s 
to w have to be estimated from analysis of variance and covariance of data in 
which the w*8 are not really known, being related to genotypes. The regression 
of x*8 on w is estimated by a generalization of the components-of-variance 
method, from variance-covariance analyses in which the usual null hypotheses 
are significantly contradicted. The net effect is that the usual significance 
tests now fail to hold, although the algebraic calculations are formally equivalent 
to those given above, once the population relations of x*a to w are established. 
When work of this kind is based on small samples, there is some diflBiculty in 
estimating the reliability of the results. 

3. Multi-dimensional discriminant functions. Instead of trying to discrimi¬ 
nate between two populations or estimate a single parameter w, our problem may 
be to discriminate among several populations, not necessarily linearly related, 
or to estimate many independent parameters Wi,W 2 ^ • • • , ly,. Just as a single 
parameter w is sufficient to distinguish between means of measurements for two 
different populations, s parameters are sufficient to distinguish between means 
of 5 + 1 different populations, and exactly s parameters will be required, if 
no linear relation obtains among the s + I populations. For example, with 
three populations, any measurement mean may be given the three possible 
values a, a -1- /3, a + 7 , corresponding to = 14^2 = 0 for population l,Wi == 1, 
W 2 = 0 for population 2, and wi ^ 0, W 2 1 for population 3. Geometrically 
we have to consider a set of parameter values as a point in an s-dimensional 
space. 

The one-dimensional discriminant function admits two very different general¬ 
izations in higher dimensions. The practical solution to a particular problem 
for which s is moderately large may involve a mixture of both generalizations. 

Let us generalize our statistical model before discussing the discrimination 
problem. To avoid complication of algebraic notation, let us for the moment 
assume a = 2. We will now postulate a set of hypothetical measurements 
& > &>••*> fp > with 


+ fiiu + yiv + €i 
f2 = aa + ftw + 72V + €2 
f 8 = 08 + <« 


f p — ap -b €p. 
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where the €p are independent normal deviates with unit variance, u and v are 
fixed variates or parameters corresponding to the different populations, and 
ai, « 2 , • * • , , A , ft , 7 i» and 72 are constants. Evidently 6 , • * • , fp can 

yield no information about u and v; and ^2 together contain all the information 
there is to get about u and v. As before, assume that our data will be in the form 
of linear combinations , with unknown coefficients Imn • The 

variance-covariance matrix within populations, or for fixed w, v, is still given by 
(Tmn = ^Imklnk • The mean values of the a;’s for fixed u, are given by 

E(Xm) = 'LltnnOln + (^m2^1 + lmA)u + (^ml7l + ^m272)v 

= Am + hmU + CmV, 

This model is again justifiable on the basis of Hotelling’s work. 

The first question to ask is whether we can now form two linear combinations 
of the x’s and get rid of { 3 , • • • , Jp in both, thus providing a two dimensional 
description of an individual on the basis of Xi, 0 : 2 , • • • , Xp . The answer here 
is in the affirmative, as a result of a direct generalization of the method dis¬ 
cussed earlier. If we calculate Xi = 2(r”‘”6mXn and X 2 = Sa’^’^CmXn, we are 
fortunate enough to get 

Xi = ftfi + 

X 2 = 7lfl + 7252 

with no disturbing elements from {a, • • • , fp . Assuming for now"that Xi and 
X 2 are not merely proportional, i.e. ft 72 — ft7i5*^ 0 , what do we do with Xi and 
X,? 

For fixed w, v, we have 

E{Xi) = Xa”^%inCin + UX(T^%ni>n + 

= A\ -f* B\U + Cio 

E^Xi) = Xa^'^CmO'n + UXa”*^Ctnbn + VXcr^^CmCn 
= Aj B 2 U -f- C 2 V 
and variances and covariance 

Til = = Bi 

Tu = = Cl = JBj 

Ta — X<r^^CmCn = C 2 . 

We may for example, estimate u and v by solving the equations 

Biu -f Cit; = Xi — Ai 


B2U + C2V = X2 — A2, 
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or we may set up regions in the Xi , X 2 plane for which certain decisions are 
made. For example, when classifying an individual into one of three popula¬ 
tions, we might delineate regions, as in Fig. 2 . 

Then the particular individual would be classified as coming from population 
II, or III, according to which region Xi, X 2 falls in. The individual points 
shown in the figure represent the expected values of Xi, X 2 for each of the three 
populations. No exhaustive investigation has been made for this situation, but 
some fairly obvious methods are available for constructing such regions. 

With respect to significance tests when the <Tmn ,am fhm,Cm are estimated from 
samples, the whole gamut of multivariate analysis has to be run. Tests ana¬ 
logous to (but more complicated than) F tests exist for testing the significance 

Classification Qeqions /r> STi.'SCz Piano 



of the discrimination, the significance of a subset of the x% and the significance 
of a theoretical pair Xi,o, X 2 ,o (Wilks [41], [42], [43]). 

For some purposes a two-dimensional discrimant function Xi, X 2 may be 
unsatisfactory. For example, we might suspect that j3iy2 = / 327 i (or that the 
relationship is nearly satisfied). Under these circumstances Xi is (nearly) 
proportional to X 2 , and we would like to compute the best one-dimensional 
discriminant function, even though we have started with two linear parameters 
u and V. Even if 18172 5 *^ iS 2 yi we might still ask for the best one-dimensional dis¬ 
criminant function, in order to rank our populations on the ‘*best^* linear scale. 
If we define V as that linear combination of ri, .^ 2 , • * • ,Xp which has the largest 
multiple correlation with u and v, we have generalized the simple one-dimen¬ 
sional discriminant function in a second direction. 

Before proceeding, it is useful to recognize that V, as defined above, must be a 
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function of Xi ^ ^ since Xi and X^ together contain all the information about 

u and V that can be obtained from the re’s. 

Now suppose we consider an arbitrary linear combination Y = XiXi + XjXj. 
Y correlates best with 


Xi(riit4 + ri2v) + X2(ri2W + r^v) = (Xirn + X2ri2)M + (X2ri2 + X2r22)v. 

We now have to choose Xi and X2 to maximize this correlation. This correla¬ 
tion will be maximized if we maximize the ratio of the variance of 


(XlTii + X2ri2)w + (Xir]2 + \2T23)v 

(over the distribution of u and v values) to the variance of Y for fixed u and v. 
Call the first quantity ^Si, the second 82 . Then & = Xi^m + 2X1X2T12 + 
and S\ is of the form XiVn + 2X1X2/112 + X2*/x22 where 

/^ii = TnCuu + 2riiTi2<rt** + Ti2*(r,» 

/^12 = TiiTi2<ruu + (ti 2^ + riiT22)<ru» + Ti2r220’»* 

/122 = rnCuu + 2Ti2T22<ru* + . 


Maximizing Si/S2 leads to the equations: 


Xl Til + X2 T 12 = (Xl jUii + X2 /il 2 ) 
02 


i.e. 


S 

Xl ri2 + X2 T22 = (Xj jUi2 + X2 /122) 
02 


Xi(rii — dfjLi 0 + X2(ri2 — 0/X12) = 0 


Xi(ti2 — 6H12) + X2(t22 — 6^122) =7 0 , with B = 81/82* 

It is thus seen that $ must satisfy the quadratic equation 
(tii — Still) (t 22 SfiTsd *” ( ti 2 “* Bfin)^ = 0 , 


in order for solutions Xi, X2 to exist. In general there will be two solutions, of 
which the greater corresponds to that linear combination XiXi + X2X2 which has 
greatest multiple correlation with u and v, whereas the smaller corresponds to 
that linear combination which has least multiple correlation with u and v. 
6 itself corresponds to R//{1 — R^) for the regression of XiXi + X2X2 on u, v. 

In the general case with s degrees of freedom corresponding to t( 7 i, 102, • • * , , 

there is an s-dimensional discriminant function (Xi, X2, • • • , X,), and a set of 
8 linear combinations for wWch R^/{1 — R^) is stationary with respect to 

Xl, • • •, X,. 

The 8 roots (corresponding to an equation of degree a) arranged in decreasing 
order, permit construction of the best one-dimensional, two-dimensional, • • • , 
(j8 — l)-dimensional discriminant functions. 
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Discussion of the relevant significance tests for these reduced discriminant 
functions is beyond the scope of this paper. Reference may be made to the 
work of Hotelling and Fisher. 
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NON-PARAMETRIC ESTIMATION II. STATISTICALLY EQUIVALENT 

BLOCKS AND TOLERANCE REGIONS—THE CONTINUOUS CASE 

By John W. Tukby 
Princeton University 

1. Suimnary. Wald [2, 1943 ] extended the usefulness of tolerance limits to 
the simplest multi-dimensional cases. Ilis principle is here used to provide 
many new ways of using a sample of n to divide the range of the population into 
n + 1 blocks of known behavior. The exact tolerance distribution for the 
proportions of the population covered by these blocks is extended from the case 
of a continuous probability density function to the case of a continuous cumula¬ 
tive distribution function. Such an extension is needed in dealing completely 
with multivariate cases cvc 7 i where the underlying distribution is as smooth as a 
multivariate normal distribution. 

The devices used in Paper I [1] to extend the usefulness of tolerance limits to 
the case of a discontinuous underlying distribution will be applied in the next 
paper of this series, with some extension, to extend the usefulness of these gen¬ 
eral tolerance regions to the case of a discontinuous distribution. Some of these 
results specialize into new results for the univariate case, although they do not 
seem to have any immediate practical application. 

The author wishes to acknowledge the stimulation given to his work on this 
problem by Henry Scheff^, wliose modesty has kept this paper from the joint 
authorship of papers I [I, Scheff (5 and Tukey 1945 ] and IV (not yet written). 

2 . Introduction. Waldos great contribution to the theory of tolerance limits 

was his method of successive elimination. As originally presented for a bi¬ 
variate situation it ran roughly as follows: Let {xi , yO, (0:2, 2/2), • • • , {xn , Vn) 
be a sample of n from an arbitrary'' bivariate population. The type of tolerance 
region to be used is determined by four preassigned integers, A*i, , ^8, and 

k\ . The procedure is as follows: Order the n observations according to their x 
values. Select the ki highest, and let the x coordinate of the lowest of these ki 
be Xu . Select the fe lowest, and let the x coordinate of the highest of these 
kihQXi, Discard these A;i + selected observations, and oixier the remaining 
n — ki — h observations according to their y values. Select the kz highest of 
these remaining observations, and let the y coordinate of the lowest of these kz 
be yu . Select the k^ lowest of these remaining observations, and let the y 
coordinate of the highest of these k^ be yi. The tolerance region, consisting 
of all points (a:, y), with xi < x < Xu and yi < y < yu depends on the sample^ 
and, hence, so does the fraction of the population falling in (= covered by) this 
region. Wald showed that the distribution of this fraction covered was in¬ 
dependent of the underlying bivariate distribution, so long as this latter dis¬ 
tribution had a continuous probability density function. He showed that the 

629 
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distribution was the same as that arising in the one-dimensional case when a 
tolerance region was set with the aid of ki + k2 + ks + ki observations. (Nu¬ 
merical approximation to these distributions will be discussed in Paper IV of this 
series. 

The important device in this process, and the one which makes the conclusion 
possible, is the discarding of the h + A-2 observations after they have played 
their part by determining Xi and Xu . 

We shall shortly be able to describe this procedure of Wald’s as a special case 
of a more general procedure, but we shall first go back to the simplest one dimen¬ 
sional case to explain some of our notions and terminology. 

Consider the uniform distribution from 0 to 1, draw a sample of n, and let the 
sample values, ordered according to size be , <2, • • • , fn • These n values di¬ 
vide the interval from 0 to 1 into the follo^\ing w + 1 parts (0, h), {h , (2), ••• , 
(tn-itn)i (tn , l) whicli wc shall call blocks. Since the joint distribution of the 
tx is well known, that of the lengths of these n — 1 blocks is easily found. This 
distribution of lengths would be unimportant, if it were not at the same time the 
distribution of the fractions of the population covered by the blocks. As is 
shown later, this distribution of fractions covered, or, more simply, of coverages, 
has the following properties: 

(i) the fractions covered add up to 1. 

(ii) the distribution is completely symmetrical. 

Property (ii) makes intuitive the result of Wilks [ 3 , 1941 ] that the distributions 
of the coverage of regions obtained 

(a) by removing the ki + k2 left-most blocks, 

(b) by removing the ki left-most and the right-most blocks 
are identical. The specific distribution obtained satisfies 

(iii) if the coverages are taken as barycentric coordinates on an n-simplex, 
the distribution over the simplex is uniform, 

(iv) the sum of the coverages of any k preselected blocks of the n + 1 has 
the well-known distribution 

Pr {sum of k coverages < <} 1 1 {n — k + k) 

where (n, m) is the incomplete Beta function. 

We shall call a set of blocks, derived from a sample, whose coverages behave in 
this general way a set of statistically equivalent blocks. Normally this will be 
abbreviated to se-blocks. (A precise definition is given in section 4 .) 

We shall concentrate much of our attention on all the blocks and their sym¬ 
metrical character, rather than on the tolerance region formed by deleting k 
of them, since our results will then be applicable to many other problems. 

Now we can generalize* Wald’s original procedure. Let TTi, TF2, • • • , Wn 
be a sample of n —^we shall not need to consider its distribution—^and let <pi , 
^, • • ■ , be n numerically valued functions of TV, possibly alike, possibly 
distinct, such that fpiW), ^(TV), • • • , <Pn{W) have a joint distribution. Proceed 
as follows: 
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Order the Wi according to the numbers select the Wi for which 

is largest and denote it by TF.d). The first block contains all W such that 

(2.1a) 

Discarding Wi^ , order the remaining Wt according to the values of 
and select as Tr<(2) the one giving the largest value. The second block contains 
all W such that 


<Pim < ipiiWia)). 
<P2m > iP2(Wi^2)). 


Continue this process. The mth block, for m < n will be defined by 

< <PjWi(j)), y = 1, 2, • • • , m - 1, 

(2.1m) 

<Pm(W) > 

and the (n + l)st block by 

(2.1n) (Pj(W) < (pjiWiij)), y = 2, • • • , n. 

(A graphical example of this construction is given shortly.) This set of n + 1 
blocks will be statistically equivalent whenever the cumulative distribution of 
each <pi function is continuous. 

To specialize this to the case described above, let IT be a pair (x, y) of numbers 
and let 

(i) the first ki ^’s be the oj-coordinate of Wy 

(ii) the next fe be minics the x-coordinate of W, 

(iii) the next kz<p^s be the ^/-coordinate of Wy 

(iv) the next h <p^a be mimes the ^-coordinate of Wy 

(v) the remaining ^’s be arbitrary. 

Then the first h blocks will contain all W for which 


that is, for which 


X = ipj(W) > <P}OVnj))y 


X > Xu = <«>*i(^t(*i)). 


i — 1, 2, • • • , All 


Similarly, the next k2 + h + h blocks will contain all W with 


X <xi, 

y>yuf xi<x<xu, 

y <yi, xi<x < Xu i 

respectively, and the removal of these ki + k2 + ks + h blocks leaves Wald’s 
tolerance region (plus the boundaries where x = Xu y x == xi y y ^ yu y y = yi). 
There would be no point in this more general wording, if it did not include 
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new cases of some interest. We give now, in graphic terms, an example of such 
a case. 

We deal with a sample of n bivariate observations, which we think of as plotted 
on a map so that we can use geographical language. The number n is rather 
large, and we wish to construct a tolerance region by deleting 12 blocks. We 
proceed as follows: 

Find the most northerly point, draw an Easi-West line through it, and shade 
the area North of the line. Find the most easterly point in the unshaded area, 
> w a North-South line through it, and shade the unshaded area East of the 



Fig. 1 


ine. Find fiie most southerly point, (always working in the unshaded area), 
draw an East-West line through it and shade the area South of the line. Find 
the most westerly point, draw a North-South line through it, and shade the area 
West of the line. Find the most northeasterly point, draw a NW-SE line through 
it and shade the area northeast of the line. Find the most southeasterly point, 
draw a NE-SW line through it, and shade the area southeast of the line. Repeat 
this 6 times more, choosing in succession the most southwesterly, northwesterly, 
northerly, easterly, southerly, and westerly points. The remaining points will 
now lie in an unshaded area surrounded by a polygon, which will have 8 (or 
perhaps fewer) sides. The inside of this polygon is the desired tolerance region. 
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Figure 1 shows the final result, starting from n = 25 . The practicing statis¬ 
tician is invited to tiy an example of his own with n at least 100. 

Other newly accessible cases can easily be invented by the reader, after he con¬ 
siders this example carefully. 

The use of a single W and n functions has two virtues; it simplifies nota¬ 
tion and frees the intuition, as compared with the use of n chance quantities 
Zi = ^ 

If the bivariate situation above were regarded as a 12 -variate situation, where 
the variates were, in order, (?/, re, — + — — x — y, — x + y, 

2/, ic, — 2/i ““ then the original Wald procedure with = • • • = fej = 

1 ; /c2 = A;4 = • • * = ^4 == 0 would apply to construct the same region. Yet 
even if x and y had a bivariate normal distribution, Wald's proof would not 
apply without extension. For the 12 -dimensional distribution is highly singular 
(it is concentrated on a 2-dimcnsional plane in 12-dimensional space) and there 
is no hope of a density function. An extension of Wald's result to the case 
where the 12-dimensional joint cumulative distribution function is continuous 
—^as is the case in this example when x and y have a continuous joint cumulative 
—^is clearly needed. 

When we come to deal with the case of where the cumulative needs not be 
continuous we shall meet a further difficulty, namely *‘ties". But if, as in the 
present case, the cumulative is continuous, it is easy to see that the probability 
that = fpiiWk) for any z, jy k is zero. 

3 . Terminology and notation. A quantity which has a probability distribu¬ 
tion we call a chance qvmitity (it has frequently been called a random variable). 
The term chance quantity does not imply that its values are single real numbers, 
they may be single real numbers (when we also speak of a real chance quantity), 
sets of n real numbers, or more general objects. The cumulative distribution 
function, or cumvlativej of a single real chance quantity, X, is defined by 

Fit) = PrlX < t], 

except perhaps at the discontinuities of F, We have used here the notation 
Pr[kiX)] to indicate the probability that kiX) holds, and we have followed our 
policy of using capital letters for chance quantities and the corresponding 
lower case letters for their values. 

The set of values of Wy or, as we shall say, the W-set, for which, for example 
<piW) < 3 , will be denoted by 

{W\^iW)<Z]. 

We shall wish to compute probabilities associated with one or more functions 
of a chance quantity; usually we will emphasize that these functions shall be 
measurable with respect to the probability measure underlying the distribution 
of W by asserting that they have a joint cumulative, which is defined by 

Fih yt2 , ••• ,tk) « Pr[iPkiW) < tk]y 
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(except possibly at discontinuities of F) and which does not exist unless the 
are measurable with respect to the unknown underlying distribution of W, In 
cases where we neglect to remind the reader, it is still assumed that the functions 
are measurable. 

The coverage of a TT-set, which may itself be a chance quantity, is defined by 
Coverage of /Sf = Pr 6 aSi}. 

When /S is a chance quantity, its coverage is also a chance quantity. The 
barycentric simplex (of dimension n) is the set of points in n + l-dimensional 
Euclidean space (<i, ^ , • • • , tn+i) with ti + 1 ^+ * • • + ^n+i = 1 and 0 < <«• < 1. 
The name comes from the representation of the point (^i ,&,••* , <n4i) as the 
center of gravity (in mechanical terms) or mean (in statistical terms) of the dis¬ 
tribution where a fraction U is concentrated at the tth vertex. (In order, the 
vertices are (1, 0, 0, • • • , 0 ), (0, 1, 0, • • • , 0 ), etc.) The uniform distribution 
on this simplex has an (n-dimensional) density 

n!M • • • d<n , (0 < , • • • , 4, 1 - - fe-fc. < 1), 

and the cumulative 


• • • J dtidU * • • dtn 

where the integration is over the range where 0 < ^* < Xi and at the same time 
+ fa + * • • + fa -1 ^ 1 * 


T{xi , , • • • , a’n+i) = n! JJ 


4 . The blocks determined by n values of W. We deal now with a population 
of W*8 (a probability measure m on the space T s {w?}), a family of functions 
^ > • • * > of W with a joint cumulative (measurable with respect toj/x) 
and a set of values Wi, W2, • • • , Wn , (wi€ T), 

(4.1) Definition The set wi ^ W2 Wn and the functions ^ i • • • , 
define blocks as follows: 

( 4 . 2 ) = {w\ <pi(w) > Oi} 
where Oi = max <pi(wi) = ^i(iy*a)), which defines t(1)» 

i 

( 4 . 3 ) & = {ty I <Piiw) < ai , v^(w?) >02}, 

where Os = max <p2(Wi) == e(2) 9^ t(l)f which defines z( 2 ). And in gen-- 

WtX) 

eralf for 1 < k < min (m, n), 

( 4 . 4 ) Sk = {M^ki(w;) < ui, ••• , < a*-j, (pk{w) > a*}, 



NON-PARAMETRIC ESTIMATION 


535 


where au = m^' ipk{w^ = v»*(w^*(Jb)), the maximum being taken over all i except 

t(l), t*( 2 ), • • • , t(A; — 1); and i{k) being chosen distinct from all z(;), j < k. 

If m > n, then 

(4.5) Sn+l = {w I <pi(w) < Oi , • • • , tpn(w) < On}. 

If m < n, then 

(4.6) Sm\n+l = {w I <pi(w) < Oi, • • • , < an,}. 

The result of this definition is to use Wi y • • • , Wn and , • • • , to define 
n + 1 blocks (one more than there are w*s) in case there are enough functions, 
and, in case there are not enough functions, to define one small block, St , for 
each function plus one large remainder Sfn\n +\. We notice 

( 4 . 2 ) Remark. The blocks of ( 4 . 1 ) are well defined unless tpi{wj) = fpi{wi) for 
some iy jy k. 


5. Statement of results for the statistician. The central results can be stated 
as follows: 

(5.1) Theorem i4»»|n+i . If Wi y W 2 y • • • y Wn are a sample of n from a dw- 
tributiony if (pi y (p2, * <Pm t (m < n)y are m functions such that 

<PlW)y • • • , V?m(W) 


have a joint distribution which has a continuous cumulativey and if the blocks 
Sly S2y * • • f Sm and ^min+i are defined as in (4.1), then 

(i) the blocks are disjoint chance setSy uniquely defined with probability one, 

(ii) the distribution of the coverages 


and 


Ci = Pr{w in t = 1, 2, • • • , w 


Cm\n+l = Pr {w in Syn\n-^i} 


is the same as that 0/ , fa, • • • , and U+i + fm+a + • • • + fm+i where U 

are uniformly distributed on the barycentric simplex with n + 1 vertices. 
Conditions ( 5 . 1 i) and ( 5 . 1 ii) are the precise definition of a partial family of 
statistically equivalent blocks of type n + 1 and an associated (w | n + 1) tolerance 
region. 

( 5 . 2 ) Theorem Bn+i. If Wi y W2 y • • • y Wn are a sample of n from a disiribu- 
tion, and if <pi , <p2 1 • • • , (Pm j (m > n)y are m functions such that 

^i(W), ^(W), • • •, ^m(W) 


have a joint distribution which has a continuous cumutalive, and if the blocks 
Sly S2, , Sn+i are defined as in (4.1), then 

(i) the blocks are disjoint chance setSy defined with probability one. 
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(ii) the dislributian of the coverages 

Ci = Pr [w in aS,}, t = 1, 2 , • • • , n + 1 

is the same as that of h ,t2 t • • • , tn+i , where the U are uniformly distributed 
on the barycentric simplex with n + 1 vertices. 

Conditions ( 5 . 2 i) and ( 5 . 2 ii) are the precise definition of a complete family of 
statistically equivalent blocks. In Paper III we shall have to widen these notions 
a little, and this form will then be qualified by the phrase “in the narrow sense’’. 

6. Statement of results for the measure theorist. The construction of (4.1) 
maps the product T" X C/" into where T is the set of w^s (and hence P" is 
the set of ordered n-tuples of ly’s), U is the space of all real-valued functions 
defined over T, measurable with respect to a fixed probability measure and 
possessing a continuous cumulative, (i.e. p{{w | ip{w) = c}) = 0 for all real c), 
and hence W is the space of ordered n-tuples of such functions, and is 
Euclidean n-dimensional space. More precisely, the mapping is into the bary- 
centric simplex with n + 1 vertices, a subset of and is well defined except 
for a set in T” of measure zero with respect to /x”, the power measure of /z. In 
these terms, we may restate theorem B as follows: 

( 6 . 1 ) Theorem Bn+i . Hold the n functions ip\ ^<P2 ^ • • • , and the probability 
measure fixed^ then T" is mapped into Bn and the power measure /x” is carried by 
that mapping into a measure on Bn . This measure is always n\ times Lebesgue 
measure. 

7. Wald’s principle. The essential principle behind Wald’s process of dis¬ 
carding observations is sufficiently fundamental to warrant a name of its own. 
It can be stated, quite generally, in the two following forms: 

( 7 . 1 ) Wald’s Principle, (discrete form.) Let W be a chance quantity^ 
and consider samples of n. Fix disjoint w-sets Ai, A2 y • • • , t B. Consider 
those samples of n for which exactly one value falls in each Ai and the remaining 
n-m fall in B, The distribution of the n-m falling in B is that of a random sample 
of n-mfrom the distribution of W restricted to B. (i.e. hb{X) = [n{B)]~^tJL{BX),) 

(7.2) Wald’s Principle, (conditional form.) Let W be a chance quantityy 
and q> a function such that each value of (piW) has probability zero. Consider 
samples of n. Then the conditional distribution of the w^, given that 

max <p(wi) = a, 

i 

is that of one Wio with ^(tOw) = a and a sample of n—I other Wi from the distribu¬ 
tion of W restricted to E — | (p(w) < a], 

( 7 . 3 ) Central Lemma. Let W be a chance quantity and let tpn be 

functions with a joint cumulative such that <Pt{w) = a has probability zero for each 
i and a {i,e, the joint cumulative is continuous). Then the conditional distribu- 



NON-PARAMETRIC ESTIMATION 


537 


tion of the remaining n—k after k blocks ha/oe been chosen according to (4,1) 
is that of a sample from the distribution of W restricted to 

J5 = (w; I ipi{w) < ai, • • • , ipk{w) < 

where A; = 1, 2, • • • , n. 

The proofs of these statements are elementary and direct. To establish (7.1) 
we have only to show that given two sets in 5"“*, their probabilities on the 
assumption that one Wi is in each are in the ratio of their probabilities for an 
unrestricted sample of n—A:. But the probability of finding the n—fc i/;,* in a 
set contained in and one Wi in each Ai,\s exactly 

• • • fJi(Ak) 

times the probability that n — kwi^ known to be in will fall in R. This 
establishes (7.1). 

In order to prove (7.2) we must show that the probability of a set R of n- 
tuples , 1 ^ 2 , • • * , t£?n is the same whether calculated directly or calculated by 
the proposed conditional distribution. To this end, it is natural to decompose 
R as follows: 

R = R{1 ) + R{2) + • • • + R{n) + Z, 

where R{i) contains those (u^i, • • • , t^n) in R for which <p{w^ > fp{Wj) for all 
j 9 ^ iy and Z contains the remaining (te?i, • • • , ; which must involve at least 

one tie (p{wj) = <p{wk)y j k. Since Z has probability zero, it will sufl[ice to 
establish the equality of the two calculations for sets of the form R{i)y and be¬ 
cause of symmetry we may restrict ourselves to sets of the form /?(!). 

Given an integer N, we decompose the range of (p{w) into Nn segments of equal 
probability, which we may do because the cumulative of <p is continuous. There 
are then Nn values 6* , (6o == — <», = + «>) such that 

Pr {bk^i < <piw) <bk} ^ 1/Nn. 

We now decompose our set R (which is of the form /2(1) as follows: 

== 722 + ' • * + Rsn + y > 

where Rk contains those n-tuples 

(t^i, • • • , t^n) for which bk^i < <p{wi) < 6* 

and ipiwi) < bk^i for all t > 1. The remaining set Y contains n-tuples where 
the two largest <p(wi), (i ~ 1 and i = O, belong to the same interval. The 
probability of this is less than 

n(n - 1) / 1 Y ^ 1 

,nN/ - 2N^ 


2 
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as calculated from the known distribution. Calculating from the conditional 
distribution, we find immediately a bound of 




n — 1 _ n — 1 k" —• (Jo — 1)* 

~k (77^ ^ k 







where is a constant depending only on n. Thus, as N increases, the prob¬ 
ability of the successive sets Y tend to zero—calculated either way. To show 
the equivalence of the two calculations it is now sufficient to show that they 
agree for the sets Rk . But this is a case of (7.1) and the lemma is proved. 

Now (7.3) follows by induction, applying (7.2) at each step. 


8. Proof of theorems. We notice that Theorem is equivalent to Theorem 
Afn\n+i , since, according to (4.1) Sn\n+i = Sn+i . 

We have only to prove theorem Afn\n+i , which we do by induction on m. 
For m = 1, it is exactly Wilks’ [3, 1941] original one-dimensional theorem, and 
is known. I^et us assume it for m = A; and demonstrate it for m = fc -f- 1, for 
by induction this will complete the proof. 

We must deal with the blocks , <Si 2 , * • • , *5* , Sk+i and >Si*+iin+i, (notation 
as in (4.1) and (5.1)). We need the obvious 

(8.1) Lemma. Since the cumulative of is continuous, the union of Sk+i 
and & 4 i|in+i differs from &(mm by a set of zero 'probability. 

Hence _ , 

CA'|n41 ~ Ca+ 1 -h CA;4i|n+l . 

Since we know from the induction hypothesis that ci, C 2 , • • • , c* and CA:|n+i 
have the correct joint distribution, we have only to show that Ca+i and Ci , 
C 2 , • • • , Ck have the correct joint distribution. Fix ci , C 2 , • • • , ca . Then 
, 02 , * • • ,0a must be fixed, and so (7.3) applies to the n—k 'WiQ not dis¬ 
carded after Oi, 02 , • * • , Oa have been fixed. The conditional distribution of 
ca +1 must be that of a fixed number (1 — Ci — C 2 — • • • — Ca), which is the 
probability attached to >8Ain+i, times the coverage of one block based on a sample 
of n—A:, since the remaining n—A; i^’s behave like a sample. 

Consider the very particular case where w is uniformly distributed between 
zero and one and ipiiw) s w, all that we have said in the last paragraph applies 
—the conditional distribution of ca+i given Ci, C 2 , • • • , ca is the same in the two 
cases—hence the joint distribution of Ci, C 2 , • • • , ca , ca+i is the same in both 
cases—but in this very particular case the joint distribution is known to be 
that required by theorem Ak^i\n+i • 
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SOME BASIC THEOREMS FOR DEVELOPING TESTS OF FIT FOR 
THE CASE OF THE NON-PARAMETRIC PROBABILITY 
DISTRIBUTION FUNCTION, I 

By Bradford F. Kimball 
State Department of Public Service^ New York, N, F. 

1. Summary. In developing tests of fit based upon a sample On{xi) in the 
case that the cumulative distribution function F(X) of the universe of X*s is 
not necessarily a function of a finite number of specific parameters—sometimes 
known as the non-parametric case—^it has been pointed out by several writers 
that the ‘‘probability integral transformation*' is a useful device (cf. [l]-[4]). 

The author finds that a modification of this approach is more effective. This 
modification is to use a transformation of ordered sample values x, from a random 
sample 0»(a;,) based on successive differences of the cdf values F(xi). 

A theorem is proved giving a simple formula for the expected values of the 
products of powers of these differences, where all differences from 1 to n -j- 1 are 
involved in a symmetrical manner. 

The moment generating function of the test function defined as the sum of m 
squares of these successive differences is developed and the application of such 
a test function is briefly discussed. 

2. Introduction. Let the sample values Xi be ordered so that 

(2.1) Xi ^ Xi^i , (i = 1, 2, • • • , n — 1). 

Let Fr denote the value of the cdf F(X) associated with the rth ordered sample 
value Xr . Thus 

(2.2) Fr = F(Xr). 

Consider the following transformation of the ordered sample values Xi based 
upon the (hypothetically) known cumulative distribution function F{X) which 
will be taken as a continuous function of X over its admissible range: 

ui = Fi, 

(2.3) Ur = Fr - Fr-i , (r = 2, 3, • • • , ll) 

Un+l = 1 - Fn . 

The restrictions on F< are that 

(2.4) Fi ^ Fi+i , and 0 ^ Fi ^ 1. 

The above transformation (2.3) translates these conditions into the symmetrical 
conditions 

(2.6) 0 ^ Ui, and ^1 + ^ 2 + * * * + + Un+i = 1. 

A one-to-one correspondence between Ui and F< exists if one of the be omit¬ 
ted,—say Up . With Up omitted, the Jacobian of the transformation from Fi to Ui 

640 
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has value unity. The probability density of the sample On{xi), with Xi ordered, 
is given by 

(2.6) P[On(a:,)] dOn = nl dFt dF* • - • dFn . 

Hence with omitted, 

(2.7) F[On(a;<)] dOn = n\dui du% • • • dup-^i du^^i • • • dWn+i. 

The sample space of the Ui with omitted, is that portion of the n + 1 
Euclidean space of all the Ui variables, bounded by the coordinate hyperplanes, 
which is on the projection of the hyperplane (2.5) upon the hyperplane = 0. 
This is a region in the ?i-space of the w*- with Up omitted, bounded by the coor¬ 
dinate hyperplanes and the hyperplanc 

(2.8) Ui + U2 + •' • + Up-i + up+i -f* * * * + + Un+l = 1. 

Thus the formal integral of the pdf of the over sample space is 

( 2 . 9 ) ^* / ^ / ‘ * * dup^i • • • dUn+i = 1 

with 0 ^ , and Ui bounded above by the hyperplane (2.8). 

It is now clear that both the pdf and the sample space of the Ui (with up 
omitted) are symmetrical in the Ui . This fact leads to complete symmetry of 
the joint distribution function of any set of Ui , over i = 1 to n + 1 including up , 
relative to the Ui selected. Other interesting results are forthcoming. 


3. Basic mathematical theorem. Using the techniques associated with the 
Beta function, the expectation of the products of powers is found to be 

E[uf^u\ •<•.•] 

(3.1) 

= r(n+ l)r(p+ l)T(q+ l)r(u;+ 1) ••• /T{n + p + q + w +-f-1) 

where r, «, f, etc., are any set of different indices (for the present other than fi) 
from the integers 1 to w + 1, and p, q, Wy etc., are any real numbers greater than 
minus one. The relation (3.1) can further be generalized to the case where Up 
may be included. This* will be proved for the case n = 2, with p, q and w 
taken as integers. The generalization can be concluded from inspection. Thus 
with 


Eluf^ulujf] = 21 


21 


Ws == 1 — — W2, 

I ul dU2 I Ui(l — Ri — U2)^ dui 

Jo Jo 

f m|(1 - jf ‘ B'a - s)" da 


2\p\w\ 


{p + v) + 1)1 


f t^Kl — Wz)*"^"*** dwj « 7— 

Jo (p 


2lplqlwl 


+ q + w + 2)r 


Hence the theorem: 
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Theorem. Given a random sample of n values of X from a universe with cdf 
F{X) which is continuous over the range of X. With the sample values Xi ordered 
so that Xi ^ Xi+\ define a set of n + 1 variables Ui as the successive differences of 
F{Xi) by the relations (2.3). The expected value of the product of real powers greater 
than minus one of any or all of the Ui, (i = 1,2, • • • , w + 1), is given by the rela- 
tion (3.1) above {not subject to the omission of uf). 

There are many.interesting consequences of this theorem. Perhaps the most 
striking is the following: 

Corollary 1 . Let a range aim^ k) for positive integer m be defined by 

(3.2) aim, k) = Fixk+m) - F{xk) 

with A; = 0, 1, 2, • • • , n, and m ^ n + 1 — A; 

under the convention 

Fixo) = 0 , F{Xn+\) = 1 . 

The probability distribution of a(m, k) is independent of k and hence is the same as 
that of F(xtn)- 

Another interesting consequence (not new) is the following: 

Corollary 2. The correlation of xu and Uk, i ^ k, is the same for all pairs 
(t, k) over the range of indices from \ to n + 1 , and has the value — 1 /n. 
Introducing the notation 

(3.3) [n + r]r - (n + r){n + r - 1 ) • • • (n + 1 ), 
the corollary follows from the relationships 

E{ui) = l/(n + 1), E{u\) == 2 /[n + 2]2 , E{ucUk) = i/[n + 2]2. 

The fact that the correlation between any two frequency differences Ui and Uk 
is negative leads to the following more general relationship: 

Corollary 3. For any set of different indices i, j, k, etc., and for any positive 
numbers p, q, r, etc., the expectation of the prodxict of the powers p, q, r, • • • of 
Ui, Uj, is less than the product of the expectations of the powers taken 

separately: 

(3.4) E[ur-u)-ul . • .J < Eiun-E{u))-Eiul) • • • . 

This follows from generalization of the relation 

Tin + i)r(p + I)r(g + I)r(r + D 
r(n + p + g + r + 1) 

[r(n + i)]*r(p + i)r (9 + i)r(r +1) 
r(n + p + l)r(n + 3 + l)r(n + r + 1) • 

The above theorem suggests the possibility of test functions for fitted distribu¬ 
tions, relative to a universe with a cdf which, since it is merely conditioned by a 
sufficient hypothesis for the theorem, may be of the non-parametric type. 
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A test function of the form 

(3.5) ^ ~ V ^^8,1 and positive 

m 

might first come to mind. If p = 1, compensatory effects of deviations reduce 
the efficiency of the test function. One is thus led first to consider the test 
function (3.5) for the case p = 2. 

4. The moments of the probability distribution of ijm = 2 u?. We are 
first concerned with the problem of the determination of the moments of the 
function 

(4.1) J/m = 2 

m 

where i ranges over any particular fixed set of m integers which for simplicity 
is usually taken as the first m. 

One first recalls the fact that the result is independent of which m indices have 
been selected; and that the expected value of any combination of powers is 
independent of which specific subscripts of u, are involved. 

Since the Ui are correlated, principles of combinatory analysis are involved in 
determining the moments of . One possible way of obtaining the moments 
is as follows: 

Let Vr denote the rth moment of ym about ym = 0. Thus 

(4.2) EKy^'i =vr^ 2?1(E 

m 

Now in the expansion of (J^ > the sum of the power indices of each term 

m 

is 2r. Thus referring back to (3.1) and (3.3) it will be noted that the expected 
value of each such term will have the common factor 

\/[n + 2r]2r. 

Consider a general term of the expansion of (]^ u<) 

m 

• • • wj**, with ^ + r 2 + * * * + T* = r. 

Clearly 

• • • uX!*) = 2 ri ! 2r2 ! • • • 2 rk \/[n + 2r]2r. 
and the coefficient Crir,. r* is the multinomial coefficient 

r! 

Crirt-r* - • 

Now in the expansion of (X^u^)'’ group the terms which have the same set of 

m 

k values of r<, irrespective of which indices of Ui are involved. The number of 
such terms (since each involves k different indices) is (?). If n , ^ 2 , • • • , , 
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are all different each combination could be taken in k I different ways. Thus with 
r’s all different and fixed, the sum of all coefficients of terms with same combina¬ 
tion of 2r,- powers (irrespective of variation of indices of the t*,) is 



This would then constitute the total multiplier for 

2 ri ! 2r2 ! • • • 2 rjk !/[n + 2r]2r 

for a given set of k r*s which are all different. 

If some of r^s are repeated, let Aji , fe , • * * yk, denote the number of repetitions 
of each different r,- (ki ^ 1, and h + k2 +••• + kg = k). Then each com¬ 
bination of the k r’s corresponding to a set of k products could be taken in 

kl/{ki\ fe! ••• k,f) 

different ways. Hence the lemma: 

Lemma 1. Consider all admissible sets of k different subscripts of Ui and a fixed 
set of values of r = ri ^ r^ rk where 

n + r2 + • • • + rjk = r 

such that s of these r *8 are different, and the number of repetitions in the set of r^s is 
given by ki h ••• kg (ki ^ 1, and ki + k2 + • * • + kg ^ k). The composite 
coefficient of the terms in Vr involving the factor 


is given by 
( 4 . 3 ) 


2ri! 2r2l • • • 2 rk\/ln + 2r]2r 

(m\ kl r! 

\A;/ A;iIA?8! • • • ri\rz \ • • • rk\* 


Examples of computation of Vr by means of the above lemma, 
moment is given by 

( 4 . 4 ) vi =?= = m 2 \/[n + 2]2 • 

m 


The first order 


The second order moment is given by 

= CrE(,u\) + 

m 


and determining the values of C,- from Lemma 1, 

+ (2) (II) ini [n + 4 U 


( 4 . 6 ) t;, = [m 4 ! + 8 (2)]/- [m + (3) j]/(” J ^) • 
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Again for the third order moment, 

•>» = = CiE(ui) + CoECuy,) + CtE(.u*tu)ul), 

tn 


and using Lemma 1, 

- h' + (”) iTii ira*'"' + (J)ll rmri'“'®="]/>” + «• 

- |^m61 + ^ 2 ) 2!3!4! + ^ 3 ) 2!2I2!3!j/ [n + 6], 


or 


(4.6, 

Similarly writing the fourth moment in the form 
W4 = Ci^(w*) + CMu)) + CzEiuW;) + C«£(m 5 u?mJ) + CMu)uW.) 
and using Lemma 1 it reduces to 

(4.7, + (”)2 + (”)| + (J)| + (:) i]/(” + >). 

. Higher order moments of the probability distribution function may be com¬ 
puted as desired. 

An alternate method of computing the moments of the distribution of this test 
function is the following: 

Consider a function goix) such that 

(4-8) = (2r)!, !7o(0) = 1. 


Thiis 

(4.9) E[u^'\ = [drgo(.0)/dx']/ln + 2r]„. 


From the principles of combinatory analysis of linear operators, it follows that* 


(4.10) 


Ei(z: «!)i - 


/[n + 2r]2r. 


Although this is an enlightening analytical form, actual computations seem to be 
simpler with the use of Lemma 1. 


^ One way of seeing this is to first think of the Ui as statistically independent. The 
numerators of the resulting terms would be the same as in (4.10). When the m are taken 
as dependent, by virtue of (3.1) the numerators will remain the same while all denominatoni 
will reduce to [n + 2r]tf. 
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Moment generating function. The moment generating function of the prob¬ 
ability distribution of ym can be written as 

(4.11) Eie'^') = God, m) = 1 + f: l<f(go(x)r/dx' | «„]/[« + 2rU'/rl 

r -1 

with 

goix) = 1 + 2 ! X + 4 ! x^/2\ + 6 ! x^/3l + • • • + (2r) I x^rl + • * • 

[n + 2 r] 2 r = (n + 2 r)(n + 2 r — 1) • • • (n + 1). 

Although go{x) exists only as a formal power series, Go{ty m) is defined by (4.11) 
as a power series with positive coeflicients, converging for all t. 


5. Some comments on test function, p = 2 . At the present time the study of 
the test function for p = 2 has not gone far enough to justify publication of re¬ 
sults. One difficulty is that although its asymptotic distribution function ap¬ 
pears to be normal, the convergence towards normalcy may be extremely slow 
in some cases. 

Furthermore there are indications that the case m = n + 1 will give the most 
definitive results not only because the complete range of data is used, but also 
because errors of Type II would in general have a less erratic effect. 

For the case m = n + 1 the mean, variance and third and fourth reduced 
moments (i.e. moments about the mean divided by corresponding power of <r) 
are: 

Case m = n -f 1 . 


( 6 . 1 ) 


E{y n+i) = 2 /(n + 2 ), a = 4n/[(n + 2 )’(n + 3)(n + 4)], 
lOw - 4 ^^/ (n + 3)(n~T T) 


<X3 


\kz/o 


(n -f 5)(n + 6 ) 


0(4 


r n* + lOln* + 14n — 8 1 

r3(n + 3)(n + 4)n 

L(n + 5)(» + 6 )(w + 7)(w + 8)J 

L n J 


_ 6(41n" + 241n + 118n^ - 784n - 48) 

n(n 4- 5)(n + 6 )(n 7)(n -f- 8 ) 


If data is not grouped the test may be applied as follows: Given a function 
Q(X) which has been fitted to the cdf F(X). From a random sample of size n 
with Xi ordered as in ( 2 . 1 ) compute the successive differences of Qixi) to obtain 
the variables . Then consider the sum of the squares 

17* = E uV. 

n+l 

If Q(X) is a true representation of F(X) the variation of U* will follow that of 
^n-fi. Thus the expected value of U*, its variance etc. will be independent of 
the fitted function Q(X), which represents certain advantages over the x test. 
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The effect of Type II errors can be roughly analyzed as follows: In considering 
the effect of such errors the testing procedure must be criticized from the point 
of view that 

Q(X) 9^ F(X). 

For m = n + 1 it still is true that 

S w* = 1 

which tends to act as a control upon U*. For example set 

Ui = Wt + x< • 

Then from the above relation it follows that 

(5.2) 2x.- = 0. 

Write U* as 

U* = 2u] + 2x? + 2Zuai 

(5.3) 

= + 2x** + (22xt)/(w- + 1) + 22xt5(Wt) 

where S{ui) denotes the variation of the true frequency differences from their 
expected value l/(n + 1). 

The variation d{ui) will be to a considerable degree independent of x<. Thus 
the term 2xi will in general tend to be larger than the last term on the right. 
The third term on the right will be zero by virtue of (5.2), and hence U* will tend 
to be larger than yn+i . A similar effect upon the sampling variance of U* can 
be noted. Hence an interval of rejection 

U* ^ A, ^ A] = a = confidence level, 

is pointed to. 

On the other hand if m < n + 1 the condition (5.2) no longer holds, the term 
(2 Zxi)/(n + 1) of (5.3) will not be zero and in many cases would dominate the 
other two error terms. Thus it is easily conceivable that one may have in the 
case m < n + \ 

Ul <yn, 

even when the discrepancies x< are large. Hence in the case m < n + 1 choice 
of confidence interval will require considerable care (see [1]). 

Although the distribution of y^+i for small n is decidedly non-normal, if the 
test function is replaced by 

(5.4) r„+, = (2[m< - l/(» + 

it will be found that the probability density function takes on the normal charac¬ 
ter quite rapidly with increasing w. Indeed the author has found that a com¬ 
puted approximation to the probability density function of Vn+i with n = 4 is 
decidedly normal in character. 
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AN ESSENTIALLY COMPLETE CLASS OF ADMISSIBLE DECISION 

FUNCTIONS 

By Abraham Wald 
Columbia University 

Su mm a r y. With any statistical decision procedure (function) there will be 
associated a risk function r{0) where r{Q) denotes the risk due to possible wrong 
decisions when 6 is the true parameter point. If an a priori probability distribu¬ 
tion of B is given, a decision procedure which minimizes the expected value of 
r{B) is called the Bayes solution of the problem. The main result in this note 
may be stated as follows: Consider the class C of decision procedures consisting 
of all Bayes solutions corresponding to all possible a priori distributions of B, 
Under some weak conditions, for any decision procedure T not in C there exists 
a decision procedure T*mC such that t*{B) ^ r{B) identically in B, Here r{B) 
is the risk function associated with T, and r*(0) is the risk function associated 
with T*. Applications of this result to the problem of testing a hypothesis are 
made. 


1. Introduction. In some previous publications [1], [2] the author has 
considered the following general problem of statistical inference: Let 
X = (Xi , • • * , Xn) be a set of chance variables. Suppose that the only infor¬ 
mation we have concerning the joint distribution function F of these chance 
variables is that F is an element of a given class U of distribution functions. 
Suppose, furthermore, that a class D of possible decisions d is given one of which 
is to be made on the basis of an observation a: = (xi, •»• , Xn) on the chance 
vector X, The problem is then to construct a function d{x), called statistical 
decision fimction, which associates with each sample point x an element d[x) 
of D so that the decision d{x) is made when the sample point x is observed. A 
statistical decision function d{x) is defined over all possible points x of the sample 
space and for each sample point x the value of the function is an element of D. 
Each element d of D will usually be interpreted as a decision to accept the 
hypothesis that the unknown distribution F of X belongs to a certain subclass 
ci> of U. Different elements d of Z> correspond to different subclasses c*> of U. 

The problem of testing the hypothesis H that the unknown distribution func¬ 
tion F belongs to a given subclass ta of U, is contained as a special case in the 
above general problem. The space D will then contain only two elements, 
di and dk , where di denotes the decision of accepting H and dk denotes the 
decision of rejecting H. 

As in [1] and [2], we shall assume also here that U is a A;-parameter family of 
distribution functions. Then each element of U may be represented by a point 
^ s= (01, ... , called parameter point, in the A;-dimensional Cartesian space. 
The class Q is then represented by a subset of the A;-dimensional Cartesian space, 
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called parameter space. We shall, therefore, refer to Q as the parameter space 
and to its elements as parameter points. 

The merits of any particular decision function d{x) will usually depend on 
the relative importance of the various possible errors caused by not selecting 
the proper element d of Z). The relative importance of such errors has been 
described in [1] and [2] by a weight function W(6, d) defined over the product of 
0 and Z>. For any pair (0, d) the value of W(df d) is non-negative and expresses 
the loss caused by taking the decision d when B is the true parameter point. 
For any given decision function d{x) the expected value of the loss is given by 

(1.1) r{e) - [ W[e, d(x)] dF{x) 

Jm 

where M denotes the sample space and F(x) is the joint cumulative distribution of 
X = (Xi , • • • , Xn) corresponding to the parameter point B. 

The function r(B) is defined over the parameter space Q and is called the risk 
function. The shape of the risk function r{B) will, in general, be affected by the 
decision function d(x) used. To put this dependence in evidence, we shall use 
the symbol t[B | d{x)] to denote the risk function r{B) associated with the deci¬ 
sion function d{x). 

A decision function d{x) is said to be uniformly better than the decision 
function d*{x) if 

(1.2) r[B I d(a;)] g r[B | d*(x)J 

for all B and if there exists at least one point B for which the inequality sign holds 
in (1.2). A decision function d(x) is said to be admissible if no other uniformly 
better decision function exists. 

A class C of admissible decision functions will be said to be essentially complete 
if for any decision function dix) not in C there exists a decision function d*{x) 
in C such that 

r[B 1 d*{x)] ^ r[B | d{x)] 

for all B, 

In section 2 we shall formulate certain assumptions which will then be used 
in section 3 to derive an essentially complete class of admissible decision func¬ 
tions. In section 4 applications are made to the problem of testing a hypothesis. 

In a recent paper Lehmann [3] obtained an essentially complete class of 
admissible tests for each hypothesis H of a, certain restricted class of simple 
hypotheses. The restrictions imposed on Q in Lehmann’s paper are essentially 
those formulated by Nejmian [4], [5] to insure the existence of the type Ai 
(uniformly most powerful unbiassed) test. Our definition of an essentially com¬ 
plete class of admissible decision functions agrees with that given by Lehmann 
when the problem is to test a hypothesis and the weight function W{Bjd) can 
take only the values 0 and 1. 
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2* Assumptions. Throughout this paper we shall make the following as¬ 
sumptions: 

Assum/ption 1: The parameter space 0 is a bounded and closed subset of a 
finite dimensional, say A:-dimensional, Cartesian space. 

We shall introduce the following convergence definition in the space D: a 
sequence {dm}, (w = 1, 2, • • • , ad inf.), of elements of D is said to converge 
to the element d of D if 

lim W(d, d J = Wid, d) 

tnmtto 

uniformly in 6, 

Assumption 2: The space D is compact and, for any dyW{d, d) is a continuous 
function of 0, 

Assumption 3: For any point ^ of D the joint distribution function of 
X = (Zi, • • • , Xn) admits a density function p{x, 6) for all points x of the 
n-dimensional Cartesian space M (sample space). The density function p{x, 6) 
is assumed to be continuous in x and B jointly. 

In what follows we shall mean by a distribution function f{6) of 0 a cumula¬ 
tive distribution function for which / df{B) = 1 and for which / W(Bfd)df{d) 

Jq Jq 

is not zero identically in d. 

Assumption 4* For any point x of Af, except perhaps for a set of measure 
zero, and for any cumulative distribution function /(0) there exists one and 
only one element of D for which the expression 

(2.1) f W(B, d)p(x, 6) dm 

Jq 

takes its minimum value with reSpect to d. 

Assumptions 1 and 3 in this paper are exactly the same as Assumptions 1 and 3 
in [2]. The formulation of Assumptions 2 and 4 is somewhat different from 
that given in [2]. This is mainly due to the fact that in [2] the space D has the 
same elements as Q, while here this is not necessarily so. It can be verified 
without difficulty that this slight modification of the assumptions does not 
affect in any way the validity of the results obtained in [2]. Thus, we shall be 
able to make use of any theorems proved in [2] for the purposes of the present 
paper. 

3. Derivation of an essentially complete class of admissible decision func¬ 
tions. For any distribution function f(B) defined over and for any sample 
point X let d(a;, /) denote the element of D for which the expression (2,1) takes 
its minimum value. It follows easily from the definition of r(d) and d(x, f) that 

f rlB\d(xJ)W(e) ^ [ rle\d*{x)]dm 

Jq Jq 


(3.1) 
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for any decision function d*(x). If we interpret f(d) as an a priori probability 
distribution of 6, inequality (3.1) says that the expected value of r{d) takes its 
minimum value for the decision function d{Xj /). We shall refer to d(Xy f) as 
the Bayes* solution of the problem corresponding to the a priori probability 
distribution J{B). 

We shall now prove the following theorem. 

Theorem 3.1. The class C of all Bayes* solutions dix, /) corresponding to all 
possible a priori distributions f(6) is an essentially complete class of admissible 
decision functions. 

Proof. First we show that for any distribution f{B) the decision function 
d{x, f) is admissible. Let d{x) be a decision function such that 

r[B I d(x)] ^ r[B | d{x, /)] 

for all B, Then 

(3.2) f r[B\dix)]df(B) g f rlB \ d(x, f)] df(B). 

JQ Jq 

From the definition of d(x, f) it follows that the equality sign must hold in 

(3.2) , i.e., 

(3.3) £ r[e 1 d(x)] dfie) = rie I d{x, /)] df{d). 

From the second half of Theorem 4.2 in [2] it then follows that 

r[B I d(a:)] = r[B | d(x, /)] 

for all B, Hence d(Xj f) is an admissible decision function. 

We shall now show that the class C of decision functions d(a;, /) corresponding 
to all possible a priori distributions f{B) is essentially complete. Let do{x) be 
any decision function not in the class C. The essential completeness of the 
class C is proved if we can show that there exists a distribution f{B) such that 

(3.4) r[B 1 d{x, /)] g r[B \ d^ix)] 

for all B, 

To prove (3.4) we shall consider the weight function 

(3.5) W*{B, d) = W{B, d) - r[B | do(x)] + Max r[B \ do{x)] 

The maximum of r[B | doix)] exists, since according to Theorem 4.1 in [2] r[B | do(x)] 
is a continuous function of B, Clearly, Assumptions 1-4 remain valid if we 
replace W{B, d) by W*{B, d). Let r*[B | d(x)] denote the risk function associated 
with the decision function d{x) if the weight function is given by W*(Bf d). 
According to Theorem 5.2 in [2] there exists a decision function d*{x) such that 

(3.6) Max r*[^ | d*(a;)] ^ Max r*[^ | d(x)] 
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for any decision function d{x). Since 

Max r*le I do(a:)] = Max r[fl | doW] 

® $ 

it follows from (3.6) that 

(3.7) Max r*[e | d*(x)] ^ Max rt9 | do(x)]. 

^ 9 

Inequalities (3.5) and (3.7) imply 

(3.8) r[d 1 d*(x)] ^ r[e | do{x)] 
for all d. 

For any distribution f(d) we shall denote by d*(x, f) the Bayes solution of 
the problem corresponding to the a priori distribution f(6) when the weight 
function is given by W*{e, d). Since W*(d, d) — W{B, d) depends only on $ 
but not on d, one can easily verify that d*{Xy f) = d(xy /). It follows from 
Theorems 4.4 and 5.1 in [2] that there exists a distribution /(0), the so-called 
least favorable distribution, such that (3.6) remains valid if we replace d*(x) 
by d*(Xf f ). Thus we can put 

(3.9) d*(x) = d*(xj) = d{xj). 

Hence, from (3.8) we obtain 

r[d\d(xJ)]^r[e\do{x)] 
for all 6. This completes the proof of Theorem 3.1. 

4. Applications to the problem of testing a h 3 rpothesis. In this section we 
shall apply the results of the preceding section to the problem of testing the 
hypothesis H that the true parameter point is included in a given subset« of 12. 
We shall assume that w is an open subset of 12. The space D consists now only 
of two elements, di and d 2 , where di denotes the decision of accepting H and d 2 
denotes the decision of rejecting H. 

We shall assume that the W{6j di) is equal to zero for points 0 in the interior 
or on the boundary of cu, and positive elsewhere. Similarly, TF(^, (fe) will be 
assumed to be positive for points 6 inside co and zero outside <a. For any a priori 
distribution f{d) the Bayes solution is given by the following test: We reject 
the hypothesis II if (and only if)^ 

( 4 . 1 ) f W(e, dM^y 0) dm > f W(e, d 2 )p(x, e) dfie)- 

Thus, the class C of regions (4.1), corresponding to all possible distributions 
f(6)f is an essentially complete class of admissible critical regions. 

For any critical region R we shall denote the probability that the sample z 


^ Whether the equality sign is included or not in (4.1) is of no consequence, since by 
Assumption 4 the measure of the set of points x for which the equality holds in (4.1) is zero. 
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will fall in R when 6 is true by P{B | 72). It follows from Lemma 4.4 in [2] and 
Assumption 3 that P(0 | 72) is a continuous function of B for any region 72. 
Since W{B^ di) is positive in the interior of — w, and W{B, (k) is positive in w, 
the class C of regions defined in (4.1) will have the following properties: 

(a) For any region 72 outside the class C there exists a region 72* in C such that 

P(B I 72*) ^ P(B I 72) in 0 ) 

and 

P(B I 72*) > P(B I 72) in a - w. 

(b) If 72 and 72* are members of C such that 

P(B I 72*) g PiB I 72) in w 

and 

P(B I 72*) > PiB I 72) in n - cu, 

then 

PiB I 72*) = PiB I 72) for all B. 

For any distribution giB) consider the critical region consisting of all sample 
points X satisfying 

(4.2) f p(x, e) dgie) > f p(x, e) dgie). 

Jca 

Let C* be the class of regions (4.2) corresponding to all possible distributions giB ). 
One can easily verify that any region in C is also a member of C*. Thus, the 
following theorem holds: 

Theorem 4.1 Suppose that Assumptions 1 and 3 are fulfilled and w is an open 
subset of n. Suppose, furthermore, that for any distribution giB) the set of sample 
points X satisfying the equation 

f p(x, e) dg{6) = f pix, e) dg(e) 

has the measure zero. Then, for any region 72 outside (he class C* (here will be a 
region 72* in C* such that 

Pie I 72*) < PiB 1 72) in a, 
and 

P(0.| 72*) ^ PiB I 72) in a - co. 

Addition at proof reading: After this paper was sent to the printer, the author 
obtained a generalization of Theorem 3.1 to sequential decision functions, as well as 
some other results. They will appear in a forthcoming issue of Econometrica, 
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DISCRIMINATING BETWEEN BINOMIAL DISTRIBUTIONS 

By Paul G. Hoel 
University of California at Los Angeles 

1. Summary. Given a set of k random samples, rri, X 2 , • • • , , from a 

binomial distribution with parameters p and n, it is shown that the familiar 
binomial index of dispersion 


k 



yields an approximate best critical region independent of p for testing the 
h 3 npothesis n = n© against the alternative hypothesis n > no, provided x and 
no — X are not small. Because of the nature of the test, its optimum properties 
also apply to testing whether the data came from a binomial population with 
n = no or from a Poisson population. 

2. Introduction. A problem of considerable interest in certain fields is that 
of deciding whether a set of observations should be treated as having come from 
either a binomial population or from a Poisson population. Although there was 
much discussion a few years ago concerning the best method for making such a 
decision [1], [2], [3], no solution of the problem was presented. In this paper a 
test that possesses certain optimum properties is derived for discriminating 
between two binomial populations. This test, however, is also capable of solving 
the problem of how to discriminate between a binomial and a Poisson population. 
The methods that are employed in the derivation of this test are similar to those 
of an earlier paper [4] in which the problem of discriminating between two Poisson 
populations was studied. 


3. Similar regions. Let n denote the number of trials and p the probability 
of success in a single trial for a binomial distribution. Let , xj, • * • ,Xk repre¬ 
sent the observed frequencies in k random samples from this binomial population. 
Now consider the two alternative hypotheses 


and 


Ho: n = no^p = Po 


Hi :n ^ ni> no, p ^ Pi. 


The purpose of this paper is to construct a test for discriminating between the two 
values of n regardless of the values of p; however it is convenient to begin with 
these more restrictive hypotheses 
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For the purpose of finding a critical region for testing JJo against Hi , the Xi 
will be treated as the coordinates of a point in k dimensions. The probability of 
obtaining the particular point xi, • • • , a;* when iJo is true will be denoted by 
Po [a:.]. Since the probability of obtaining x successes in n trials is given by 


it follows that 


n! 


x\ (n — a?)! 


V Q 


( 1 ) 


Po[xt\ = 


(noO* 

lb-Po‘ go' 

- Xi)\ 

1 


In searching for a critical region that will be independent of po > it is illuminat¬ 
ing to study the methods that were designed by Neyman and Pearson [5] for 
continuous distributions. These methods suggest that one should look for criti- 

ib 

cal regions on the surfaces 53 = constant. For this reason, instead of 

1 

using (1) for constructing critical regions, it is desirable to study the conditional 

k 

probability distribution of the points lying in the plane 53 where iVT is a 

1 

positive integer not exceeding km . The conditional probability of obtaining 
the point Xi, • • • ,Xk , when the point is restricted to lie in the plane ^ Xt ^ N, 

• I 

will be denoted by Po[x*-1 N], Its value may be obtained by dividing the proba¬ 
bility (1) by the probability that the point will lie in the plane ^Xi ^ N. If 

1 

this latter probability is denoted by Po[iV^], then 


( 2 ) 


PotxilAT] 


Po[Xi] 


PolNV 

Since the sum of k independent variables each possessing the same binomial dis¬ 
tribution has a binomial distribution with n replaced by kn, it follows that N 
possesses a binomial distribution and that 


(kno)l 


(3) 

If (1) and (3) are substituted in (2), it will reduce to 
Po[xi\N] 


*no-Ar 


(noO^iSriCA-no - N)\ 


(4) 


(A;no)in3?d(no - a;,) I 


This conditional probability distribution in the plane 53 ^ is independent 

1 

of Po and therefore may serve as the basis for constructing a critical region that 
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is independent of po for testing JETo against Hi . It will therefore be possible to 
test the less restrictive hypothesis 

Ho: n = no 


against 


Hi : n = ni > no . 


4. Best critical region. Although a best critical region does not exist for 
testing Ho against H[ , it is helpful to proceed as though one did. 

k 

If a critical region of size a could be selected in each plane ^ Xi = N, 

1 

(A^ = 0, 1, ■ • • , kno)y then the totality of such critical regions would constitute 
a critical region of size a that is independent of po and which therefore could be 
used to test Ho against H[. For, if Pq [X e C.R.] denotes the probability that 
the sample point, which will be denoted by X, will lie in the critical region, it 
follows that 

fenp 

Po[X € C.R.] = E Po[N]Po[X t C.R. I N] 

JV-0 


(5) 


= E PolAHa 




This last equality follows from the fact that the sample point must lie in one of 

k 

the planes ^ Xi = Ny {N = 0,1, • • • , kno). 

1 

Furthermore, this would be the only critical region of size a independent of 
po , because if a critical region of size ajv , (AT = 0, 1, • • • , fcno), were selected in 

k 

the plane ~ * •' > it would be necessary that 

1 

X) FolAHaAT = Of, 

iV-O 


independent of the value of po . From (3) this is equivalent to requiring that 


( 6 ) 


^ (kno) I 
hN\(kno - AT)! 


P0^(1 — ^ OlN 


OLy 


independent of the value of po. Since the left side of (6) is a polynomial in po, 
its constant term must equal a and all other coefficients must vanish. It will be 
observed that no terms of the sum in (6) that arise from N > r will contribute to 
the coefficient of p5 ; consequently this coefficient will not contain the unknowns 
otr+i , • * * , otkno • Thcsc Considerations show that the aif must satisfy equa¬ 
tions of the form 
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« ~ ^OOOo 

0 = Cio ao + Cn ai 


0 = CA;noOOfO + + *• * + aifcno . 

It will also be observed that €„ — (kua) [/r[(knQ — r)!; consequently the triangular 
matrix of the coefficients in these krio + 1 non-homogeneous equations is non¬ 
singular. The equations therefore possess a unique solution, namely the known 
solution of (Xff = a. 

The preceding discussion shows that it is necessary to find critical regions of 

ib 

size a in each plane ^ Xi = N, (N = 0,1, • • • , kno)y if a critical region indepen- 
1 

dent of po is desired. If each such planar critical region were a best critical 
region for that plane, then the totality of such regions would constitute a 
best critical region independent of po for testing H'q against H[ . 

It follows from the theory of best critical regions [5] that if a best critical region 

k 

in the plane ^ Xt = N did exist, it would be determined by the inequality 


(7) 


PolxilN] ^ ^ 
Pi[x,|iV] ^ 


where Pi corresponds to Po when Hi is true and where K is a constant whose value 
is chosen to make the critical region one of size a. Now from (4), 

PolxilN] _ (no!)*(A;no — N)\{kni)in{ni — x,-)! 

Pilxi I N] (ni inkrii - N) Kkuo )! n(no - x.)! ‘ 

In order to study the possibility of a best critical region, it is therefore neces¬ 
sary to study the possibility of (8) satisfying inequality (7). 


5. Approximate best critical region. Unfortunately, because the variables Xi 
are discrete, it is not possible to find critical regions of exactly size a for arbitrary 
a as required in (5). Consequently it is necessary to introduce continuous ap¬ 
proximating functions for discrete probability functions or to resort to other 
devices if critical regions of the type discussed in the preceding section are to 
be obtained. 

For the purpose of introducing such approximations, (8) will be written in 
the following form: 

Po[x< I N] ^ (kno - N)l ^ (fcni - N)l Ay"*-" 

Pj[x, I iV) n(«« — X,)! \k/ ' n(ni — X,)! \k/ * 


( 9 ) 
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where ci is independent of the variables Xi. It will be observed that the ratio 
on the right is a ratio of two multinomial functions. Now^ the multinomial 
function 


N\ 


XilX2\ •••Xkl 


*.*1 

Vl Vi ■ • • p*. 


where 23 = N, can be approximated by the multivariate normal function 


( 10 ) 


-4-4 (wDI 


The approximation is good provided the Npi are large and the Xi remain away 
from their extreme values. If this approximation is applied to both numerator 
and denominator of (9), to this order of approximation, 


( 11 ) 


PilxilN] 


Cl 


e exp 


_ 1 ^ -AT/Ar 

. ^ "r Wno - N/k 
Mkno - 



A;*^* e exp 

^‘rWni-N/kj] 


[2ir(A-»i - iV)]*«'-‘> 


kill — r 1 wi — no 

.Arno - n] " L ® (ni - N/k)(,Th - N/k) 



Since, by hypothesis, n\ > uq and no > N/k, except for the case of no = N/k, 
which will be considered later, it follows that 


__ >0 

(m - N/k){no - N/k) 

k 

As a consequence, the right side of (11) will decrease in value as 23 (^<"“ N/k)^ 

1 

increases in value. If (a;i, • • • , Xk) is a point lying on the sphere 

(12) i {Xi - N/k)* = B 

1 

and if the coordinates of this point satisfy inequality (7) when approximation 
(11) is used, then all points outside this sphere will also satisfy (7) to this same 
Older of approximation. A best critical planar region of size a in this approxi¬ 
mate sense can therefore be obtained in the plane 23 ^ by determining a 
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sphere with center at (AT/fc, • • • , N/k) such that when Ho is true the probability 
is a that a point lying in the plane will lie outside this sphere. Furthermore, such 
a region will be a common best critical region for all values of ni > no because 
the preceding arguments do not require the value of ni but merely the knowledge 
that ni > no. 

For the purpose of determining the radius of the sphere that will yield the 
desired critical region, (4) will be expressed as follows: 


(13) 


Po[Xi 


N\=e,^ (^Y 

^UZil\kJ n{no-Xi)l 


JeJ 


where C 2 is independent of the a:,*. If these multinomials are replaced by their 
multivariate normal approximations as given by ( 10 ), to this approximation 
(13) will reduce to 




(14) 


= Cs e exp 


Z) (Xi 

-i ‘ 


N/kf 


k V kno) J 


where Cj is independent of the x,-. Since 23 ^< = here, x* may be expressed 

1 

in terms of the remaining variables; consequently (14), except for a constant 
factor, may be treated as a normal distribution in the variables xi, • • • , x*-!. 
If the factorials in C 3 are replaced by their Stirling approximations, it will be 
foimd that Cs is the correct constant for the normal distribution. 

Since it is known [ 6 ] that —2 times the exponent in a normal distribution fimc- 
tion possesses a chi-square distribution, it follows that to this order of ap¬ 
proximation 


(15) 


E (x, - N/k)* 

k V W 


possesses a chi-square distribution with k — 1 degrees of freedom. If xl is a 
value such that P[x* > xil = «> then 


(16) 


i:(x. 

1 


N/k)* 


k V W 


xl 


determines a sphere such that to tUs order of approximation the probabilily is 
a that a point lying in the plane E x< = will lie outside the sphere. From 
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the arguments following (12), it therefore follows that a common best critical 
region in this approximate sense for testing Hq against H[ will consist of that 

k 

part of each plane X) (■^V’ = 0, 1, • • • , Amo), which lies outside the cor- 

1 

responding sphere given by (10). Since the Xi are non-negative and do not 
exceed no, the planes corresponding to AT = 0 and N — krio contain a single 
point; therefore it is necessary to adopt some convention that assigns 100a per¬ 
cent of the samples with AT = 0 and N = A;no to a critical region in order to obtain 
critical regions of size a in these two cases. 

For a given set of data, the procedure to be followed then consists in calcu¬ 
lating the statistic 

k 


Z) (Xi - x)* 
1 



where x = ^ Xi/k, and agreeing to reject the hypothesis that n = no in 

favor of the alternative hypothesis that n > no if and only if z > Xa , where 
P[x > Xa] = CK for A; — 1 degrees of freedom. Because of the nature of the 
approximations used in (10) and (14), this result may be expected to be accurate 
only if X and no — x are large. 

The interesting feature of this result is that the familiar binomial index of 
dispersion, z, possesses optimum properties in this approximate sense for testing 
n = Mo against n > no. 

6. Poisson application. Since the preceding test will possess approximate 
optimum properties for n as large as desired, independent of the value of p, 
and since a Poisson distribution with parameter m can be approximated as 
closely as desired by means of a binomial distribution with np = m by allowing 
n to increase sufficiently, it follows that the test will also possess approximate 
optimum properties for deciding between a binomial distribution with n = no 
and a Poisson distribution. 


7. Estimation of n. Although the purpose of this paper has been accomplished 
in the preceding sections, it is interesting to observe the role played by the closely 
related Poisson index of dispersion in the extimation of n. 

Approximate confidence limits form may be obtained by means of (16). 
If xi-a is a value of x such that P[x^ > xi-a] = 1 — a, then, to this same order 
of approximation, the probability is 1 — 2a that 

iix 
2^1 
Xl-a < - y- 

xll 
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If these inequalities are solved for n, the following 100(1 — 2a) percent approxi¬ 
mate confidence limits for n will be obtained: 


(17) 


__< n < __ 

2 _ 2(a;>- - xy 2 __ X jxi - x)^ ' 

Xa - Xl—g ;= 


Only the lower limit here will possess optimum properties. Now it will be ob¬ 
served that only positive values of n will be admissible if 

2(a;i - ^ j 

~ ^ Xi-«) 


whereas only negative values will be admissible if 

X(Xi ““ X) ^ 2 

~ x« • 


The range of values will be infinite in each case if there is equality rather than 
inequality. If, however, 


2 

Xl- 


X(xi — xf 


2 

X« 5 


then both positive and negative values of n over infinite ranges will be admissible. 
Since n increases as the Poisson index S(x» — x)*/x increases until it becomes 
infinite and then increases from minus infinity through negative values, (17) 
may still be thought of as giving an interval (infinite) of values with a positive 
“lower” limit and a negative “upper” limit. Thus, the familiar Poisson index 
of dispersion plays an interesting role in determining whether a Poisson assump¬ 
tion is reasonable as far as admissible values of n are concerned. 

If the population is truly binomial, negative values of n must be ruled out; 
consequently a Poisson assumption becomes increasingly tenable as the Poisson 
index increases. However, experience has shown [7] that a negative binomial 
distribution is often more realistic in describing data supposedly drawn from a 
binomial “or Poisson population than is the assumed distribution; consequently 
a negative binomial should be given consideration if (17) yields only negative 
values or if it yields a negative “upper” limit that is numerically small relative 
to a positive “lower” limit. 

It is also interesting to consider the point estimation of n. Here, it is cus¬ 
tomary [7] to estimate n by means of 

kx 

X 

Thus, a positive, infinite, or negative estimate for n will be obtained according as 
the Poisson index is less than, equal to, or greater than h. 
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BILINEAR FORMS IN NORMALLY CORRELATED VARIABLES 

By Allen T. Craiq 

University of Iowa 

1. Summary. If a variable x is normally distributed with mean zero, we have 
previously given a necessary and sufficient condition (see references at end of 
this paper) for the independence of two real symmetric quadratic forms in n 
independent values of that variable. This condition is that the product of the 
matrices of the forms should vanish. In the present paper, we have proved 
that the same algebraic condition is both necessary and sufficient for the inde¬ 
pendence of two real symmetric bilinear, or a real symmetric bilinear and 
quadratic form, in normally correlated variables. 

2. Introduction. In this paper, we determine the moment generating function 
of the joint distribution of two real symmetric bilinear forms in certain normally 
correlated variables and derive a necessary and sufficient condition for the 
independence, in the probability sense, of these forms. We further investigate 
tho condition for independence, in the probability sense, of real symmetric 
bilinear and quadratic forms. 

3. The moment generating function of the distribution of real symmetric 

bilinear forms. Let the two variables x and y have a joint normal distribution 
with means zero, unit variances and correlation coefficient p. From this bi¬ 
variate distribution, repeated random samples of n pairs, say (xi, 2 / 1 ), {x 2 , ^ 2 ), 
• * • » (^n , 2/n), are drawn. Let C = || || be a real symmetric matrix and write 

0 = 22 CjkX/yk . The moment generating function of the distribution of d 
is then given by 

<p<j) = £[«'*] = ( 2 ^ 

where 

Q = 2(1 - p») ^ + 

and $ is defined above. If we subject the x*s and y’s to the same linear homo¬ 
geneous transformation with appropriately chosen orthogonal matrix L, then 
Q remains invariant and $ becomes 2 yXyxj 2 /y where the X*s are the n real roots of 
the characteristic equation of C, that is, of | C — XJ | =0. The integrations 
are then easily effected and we find that 

v(.t) = {n [1 - t(p + DXylll - tip - 

= {1 / - «(p + DC I • I / - <(p - DC I }-< 

= 1 7 - 2p<c - (1 - p*)<’c’ 1 
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where I is the unit matrix of order n and the vertical bars, as usual, indicate the 
determinant of the ehclosed matrix. 

Next, let = II ajk || and B — || 6^* || be two real symmetric matrices each of 
order n. Write Bi = ZlajkXjyk and 62 = It^hjkXj'Hk where the x’s and y’s are the 
items of the sample randomly drawn from the bivariate distribution previously 
described. The moment generating function of the joint distribution of B\ and B^ 
is then given by 

♦>(<1, k ) = 

- (2irV'r^»)“’‘ /*"••■ r dyn • • • dy^ dxi, 

J —«0 J —00 

where , 0*, and Q have the meanings previously assigned to them. If we 
pursue a line of reasoning similar to that above, we find that 

^(ti , < 2 ) == I / *— 2 p(iiA + t 2 B) — (1 — p*) (tiA + t 2 B)^ I *. 

4. The independence of bilinear forms. It is clear that there exist positive 
numbers, say hi and hj, such that , U) exists for 0 < ti < hi and 0 < U < h 2 . 
It is well known that a necessary and sufficient condition for the independence 
of and Bi is that ^(ii , ^ 2 ) shall factor into the product ^(ii , 0)^(0, < 2 ). If then, 
we assume Bi and Bi to be independent, we have essentially 

I / ~ 2p((iA + tiB) ^( 1 - p^XtiA + tiBf 1 

- I / - 2p{iA - (1 - p^)t!A^ 1*1/- 2ptiB - (1 - |. 

If h denotes the smaller of hi and hi , then the factored form holds for 
0 < ti, (2 < hj and hence for all real values of h and ( 2 . In particular it holds 
for ti — h so that 

1 1 - 2ptiiA + B) - (1 - p^)t\{A + BX I 

= I / - 2ptiA - (1 - pYiA^ 1 • I / - 2ptiB - (1 - p^)tlB^ |. 

Let n , r 2 , and r < ri + Vi denote the ranks of the matrices A, B, and A + B. 
Further let the real non-zero roots of the characteristic equations of these ma¬ 
trices be denoted respectively by ai, a 2 , • • • , , jSi, ft , • • • , ft*, and 71 , 72 , 

• • •, 7 r. Then the members of the preceding equation may be written 

II [1 ~ h(p + 1)7*][1 — h(p ~ l)7t] 


and 

IT [1 — hip + l)ai][l — hip “l)at[ n [1 — hip + l)ft]ll ^hip — l)ft] 

im.1 im.1 

respectively. It is seen that the left member is a polynomial in h of degree 2r 
and that the right member is a polynomial in h of degree 2(ri + r 2 ). Accord- 
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ingly, r = n + r2 and the roots 71 , * • * , 7i consist of the roots ai, • • • , an , 
ft > • *' » *. That is, if and 62 are independent, then the rank of A + B 

is the sum of the ranks of A and B and the non-zero roots of the characteristic 
equation of A + B consist of those of the characteristic equation of A together 
with those of B. Further, if in (1) we put <2 = , where v is real, we have 

1 / - 2pti{A -1- vB) - (1 - p'XUA + vBf I 

= I / - 2ptiA - (1 - pYiA^ 1 • U - ^phvB - (1 ~ p^WB* |. 

Denote the rank oi A + vB by r' and the non-zero roots of its characteristic 
equation by 61, • • • , 6,/. The immediately preceding equation can then be 
written 

r* 

n [1 •“ hip + l)5t]ll — hip — 1)5,] 


= II [1 ” hip + — hip — l)a,] II ll “ hip + l)yft]U — hip — l)vft]. 


From this we infer that, apart from zero roots, the roots of the characteristic 
equation of A + vB are ai, • • • , ar^ , i>/3i, • • • , . 

If a symmetric matrix, say M(y), has elements which are real pol3niomials 
in the real variable v, and if the determinant 


I Miv) - X/ I = (-1)”[X - p^rnx - p 2 iv)] ... [X ~ pnW], 


where piiv), pziv)^ • • • , pniv) are likewise real polynomials in v, then there exists, 
for all real values of v, a real orthogonal matrix, say L(i;), such that 


Uiv)Miv)Liv) = 


piW 

0 •• 

0 

0 

Jhiv) 


0 


Pn(.v) 


Furthermore^ exists for all real values of v. Since 

dv 

IA -b t;B ~ X/ I = - ai) ... (X - a,,)(X - t;ft) ... (X - t;/?.,), 

^ A number of years ago, in connection with another problem, the writer sought the as¬ 
sistance of Professor N. H. McCoy for a proof that L{v) is differentiable at v ■■ 0. Pro¬ 
fessor McCoy’s elegant demonstration of the existence of L{v) showed that each element 
of this orthogonal matrix is itself a real polynomial in t», divided by the positive square 
root of another real polynomial, which polynomial is never negative and which vanishes 
for no real value of v. Thus the derivative of L{v) exists not only for t; « 0 but for all 
real values of v. The writer thanks Professor McCoy for his kind and generous assistance. 
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then A + vB belongs to the class M(v) so we have 

ai 0 * • • 0 

0 — an * * • 0 

0 • • • vfii • • • 0 

(2) L'{v)U + vB)L(v) = ‘ 

0 • • • vPn * • • 0 

0 0 


In particular, 


(3) 


ai • • • 0 


L'(0)AL(0) = 


0 0 


If we differentiate (2) with respect to v and subsequently set t/ = 0, we have 



(4) AL(0) + L'(0)BL(0) + L'(0)A^^ 

ao av 
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SiniHi L{v) is orthogonal, then I/{v)L{v) = I, Upon differentiating both mem¬ 
bers with respect to and siibsetiuentl v sotting r = 0, it is seen that L(0) = 

av 

— L'(0) fhat />'(()) is it skew-symmetric matrix, say S. Further 

(5) 7/(0) = -SL’iO), 

dv dv 

and, by taking conjugates, 

(G) = -L(0) L(0) = /.(0).<?. 

dv dv 

If we multiply (5) on the right by AL(0) and (0) on the left by 77(0)A, we see 
that (4) may be written 

110 Oil 


0 

01 


( 7 ) u(mm) - 


+ .S7/(0)AL(0) - L'(0)ilL(0)5. 


0rt 

0 


0 


0 


Since S is skew-symmetric and since 7/(0)A 7/(0) is given by (3), then each 
element on the principal diagonal of jS 7/'(0)AL(0) and L'(0)AL(0 )jS is aero. 
Further, since is symmetric, then Ij{0)BL{Q) takes the form 

0 A'i2 • • • l^ln 
kvi 0 




dr. 


k\n 


0 
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Because the non-zero roots of the characteristic equation of L'(0)/?L(0) arc , 

• * • > /^r 2 f^‘cn tlic sum of all two-rowed principal minors of the detenninant of 
Ti'(0)BL(0) must ecpial the sum of the products of jSi, • • • , /Srj taken two at a 
time. That is 

UMi = E/Sift - 

<<j <<) 

so that each kij -, being real, is zero. Accordingly, SU( 0 )AL{ 0 ) — L'(fl)AL{0)S 
is a zero matrix and I/(fi)BL( 0 ) is given by the first term in the right member 
of (7). We then have 

L'(0)AL(0)L'(0)BL(0) = L'(0)AJ5L(0) = 0, 

from which it follows that A/? = 0. Thus, if the real symmetric bilinear forms 
$1 and 62 are independent in the probability sense, the product of their matrices 
is zero. 

If, conversely, AB = 0, then 

, (,) = I 7 - 2p(M + iiB) - (1 - + iln^) 1 

= 1 [7 - 2 ptiA - (1 - Maw - 2pfeB - (1 - I 

= vih , 0)(p(0, fe), 

and 61 and 62 are independent. This establishes the following theorem. 

Theorem I. Let x and y be normally correlated with means zero, unit variances, 
and correlation coefficient p. Let Bi and 62 he two real symmetric bilinear forms in n 
random pairs of values of x and y, say (xi , yi), {x 2 , 2 / 2 ), • • • (x„ , 2 /n). A necessary 
and sufficient condition that di and 62 be independent in the probability sense is that 
the product of the matrices of the forms be zero, 

5. Simultaneous reduction of quadratic or bilinear forms. The argument 
of Section 4 may be used to establish in a very simple manner the following 
theorem. 

Theorem II. Let A and B be two real symmetric matrices with constant ele¬ 
ments, each matrix of order n. A necessary and sufficient condition that there exist 
a real orthogonal matrix of order n such that simultaneously each of UAL and UBL 
is in canonical form, wherein no non-zero elements occupy cx>rre 8 ponding positions 
on the principal diagonals, is that AB = 0. 

For if such an orthogonal matrix L exists, it is evident that UALUBL = 
UABL = 0 from which it follows that AB = 0. Conversely, if AB = 0, then v 
being a real scalar, the matrix (A — \I){vB — XZ) is equal to the matrix 
—X[(A + vB) — \I]. These matrices being equal, their determinants are 
equal so that A + vB belongs to the class Miv) of section 4. Thus L may be 
taken as L(0) and simultaneously L'A L and T/BL are of the form stated in the 
theorem. 

6. Independence of bilinaer and quadratic forms. Let B = T^XOjkXfyk be a 
real i^jrmmetric bilinear form of rank n in the previously defined variables 
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(xi, yi), ■ ■ ■ , (a’n , y„) and let q — 'S7^b,ix,xi, be a real symmetric quadratic form 
of rank »•» in Xj , a'j . As usual, denote ihe non-zero roots of the charac¬ 
teristic equations of A and B by aj, aj, • • • , a,, and ■■■ , |8,, respectively. 

The moment generating function of the joint distribution of 8 and q is 

- a wh r?)- *■ • 

where, as previously, 

“ 2(1 - p^) 

We first orthogonally transform the variables so that the exponent in the inte¬ 
grand becomes, upon writing ||/J^ || = UBL, 

hXaix'iv'i + hXT^fikXjxi - - ^ S(x;.’ -f- y'- - 2 pxW,)- 

At *“ P) 

We then inb'grato on ?/{, 7 / 2 , • • • , 2 /I, and obtain for the exponent in the inte¬ 
grand 

hX-S,f,kXiX„ - hlx'i + piauix' -f t\Xa*iXi . 

If we effect on the variables , 0 * 2 , • • • , Xn the inverse of the orthogonal trans¬ 
formation initially used on the a^’s and the exponent in the integrand becomes, 
using II ffyt II = A\ 

h' 2 :^bjkXjXk ~ 524 + p/i 22 a,jfcX/a’jb H-^ tlXT^gjkXjXk 


or 

— ^22[dyA; ^ptittf'k ~ (1 P^')^i9jk ^ttbjk]XfXk I 

where djk equals 1 or 0 according as j does or does not equal k. Hence, 

(8) <p{ti , ^) = \ I ^phA — (1 — p^)tiA* — 2tiB I ^ 

If 6 and q are independent, we have 

(9) I 7 - 2ptiA - (1 ~ pYiA^ - 2hB 1 

= I 7 - 2phA - (1 ~ pViA^ I . 1 7 - I, 

for 0 < h < hi and 0 < ^2 < ^ 2 . As before, the members of (9) are polynomials 
which, being equal for 0 < < 1 , < /i, are equal for all real values of ti and 1 %, 

If we put ti = 1 and <2 = vti = v, where v is real, then (9) becomes 

I 7 - 2pA - (1 - p')A* - 2t;5 I = 1 7 - 2pA - (1 - p*)A‘ | • | 7 - 2vB | 

= it [1 - (p - - (p + - 2f/Jy). 

1 1 
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That is, 

j 2pA “1“ (1 — "i" — X/ I 

= ( —— 2pai — (1 — p*)ai]*'«[X — 2par, — (1 — p*)a%J 


•[X - 2rAl---[X - 2vpr,] 

so that 2pA + (1 + 2i'^ is a matrix of the class M(v). Hence we write 


i 2pai + (1 p^)oci • • • 0 

• 2pari + (1 — P^)^r, 

2v0i 


U{v)l2pA + (1 - p')A' + 2vB]L(r) = 


2vprt 

0 


0 


Ol! 


The argtiment of section 4 shows that 7/(0)[2pA + (1 — p^)A^]L{())iy(i))2BL{0) 
is a zero matrix, from which it follows that 2pAB + (1 — p^) A^B = 0. But 
this imposes on p, n* conditions of the form 

2pZi* + (1 — p^)mjk = 0, 0, ^ 2, • • • , n). 


Since these hold for every — 1 < p < 1, they hold identically. Hence each /,/. 
and nijk is zero. In particular, 1| = .47? = 0 if P and q are independent. 
Conversely, if ^47? = 0, we see by 'J'heoi-em II that (8) becomes 


<p{ti, ^2) = ipih , 0)^(0, h)^ 

so that 6 and q are independent. This yields Theorem III. 

I'heoreXi III. Let x and y he normally correlated with means zero, unit vari- 
ances, and correlntion coefficient p. Let B he a real symmetric bilinear form in the 
n random pairs of values of x and y, say (xi, 2 / 1 ), • • • , {xn , 2/„), and let q he a real 
symmetric qvxsdratic form in .Ti , X 2 , • • • , o-n (or i/i, • • • , i/„). *1 necessary and 
sufficient condition that $ and q he independent in the prohahility sense is that the 
product of the matrices of‘the forms he zero. 

For example, let P be n times the sample covariance and let q ho n times the 
square of the mean of the a;’s. Then 

P » Xixj - £)(yj ~ y) 


« X2ajkX/yk ’, 
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where 


n — 1 .. . , 

Oji = —— if J = fr, 

n 

= — i otherwise, 
n 

and 

q = nx* = XXbjkXjXk , bjk ~ 1/n for j, Ar = 1, 2, • • • , n. 

Since AB = 0, then d and q are independent, a fact otherwise known but perhaps 
not so easily established. 


REFERENCES 

[1] Allen T. Craig, “Note on the independence of certain quadratic forms”, Annals of 
Math. Stat., Vol. 14 (1943), pp. 195-197. 

[21 Harold Hotelling, “Note on a matric theorem of A. T. Craig”, Annals of Math. 
StaL, Vol. 16 (1944), pp. 427-429. 



ON THE CHARLIER TYPE B SERIES 


By S. Kullback 
George Washington University 

1. Introduction. The Type B series of Charlier has been discussed in some 
detail in the literature (See references at the end of the paper). The problem 
of the convergence of the Type B series has been considered by Pollaczek- 
Geiringer [12], [13], Szego [12] (page 110), Uspensky [16], Jacob [5], Schmidt [16] 
and Obrechkoff [11]. There is presented in the following a method of develop¬ 
ment of the Type B series which is believed to be of some interest, including a 
necessary and sufficient condition for the convergence which is basically the 
same as that of Schmidt [16]. A result of Steffensen [17] is extended and shown 
to be related to the Charlier Type B series. 


2. Statement of results. Consider the function p(r), defined for r = 0, 1, 2, 
• • , and such that 


(2.1) S ?(»•) = 1; I V{r) I = ^1 

r.«0 r.a0 

where A is some finite value. Let the n-t\\ factorial moment be defined by 

M(0) = 1 


( 2 . 2 ) 


M(n) 


= ^r(r - l)(r - 2)* • • (r - + l)p(r), (n = 1, 2, • • •) 


For arbitrary \ let 

(2.3) 


j. n{n — 1) .2 

Ln = M(n) ~ n/X(n-l) A -f - — - A«(n-2) A 

_ n(n — l)(n — 2) 
3! 


M(n-3) X® + • • • -f- (— 1)“X”. 


We prove the following results: 

Theorem. A necessary and sufficient condition that the function p(r) of (2.1) 
may he expressed by the absolutely convergent series 


(2.4) 
is that 

(2.5) 


p(.) = + 


+ I M(l) I + il I M(2) I + ^1 I M(3) I + 


+ I M(n) I + 


converges where Ln is defined as in (2.3). 

574 



CHABLIBR TYPE B BERIEB 


676 


3. Generating functions. For the function p(r) of ( 2 . 1 ) consider the gen¬ 
erating function defined by 

(3.1) „(a) = 2 2'p(r) 

rimO 

where gisa complex variable. Because of ( 2 . 1 ) it is clear that the right member 
of (3.1) is uniformly and absolutely convergent for | 2 | ^ 1 so that the radius 
of convergence of (3.1) is some value Ei > 1. 

The Taylor expansion of ip(z) about the point 2 = 1 is given by 

(3.2) „(*) = ^(1) + (z- l)^'(l) + ^"( 1 ) + ... 

where, as may be readily obtained from (3.1), 

(3.3) ^'"'(1) = f: r<r - l)(r - 2)- • • (r - n + l)p(r) = . 

r-.0 

If it is assumed that (2.5) converges, then 

(3.4) ^(Z) = 1 + (2 — 1)/X(1) + - - 21 " " +*•• + — ■ M(n) + • • • 

is uniformly and absolutely convergent for | z — 1 | ^ 1 . 


(4.1) 


4. Sufficiency. For arbitrary X let us set 


= 1+ L .(2 - 1) + ^ (Z - D* + 


where the right member, because of (3.4) is absolutely convergent for 
I 2 — 1 I g 1 . The coefficients on the right side of (4.1) are given by 

(4.2) L„ = M(n) — n/x(„-i)X H—^ M(«-2)X* — • * • + (— 1)”X" 

and the factorial moments may also be expressed by 


f4.3) 


M(n) 


= Ln + nLn-i X + ^ L»-2 X* + * * * + X”. 


2 ! 


These relations are readily derived by expressing (4.1) symbolically as 

(4 4 ) 

where after expansion m** and L” are to be replaced by fji(n) and Ln respectively. 
(Of. Jordan [7], p. 39). From (4.1) and (3.4) there is now derived 


(4.5) 


^( 2 ) 


e 




(1 + M2-I)+§[(*-!)*+ •••). 
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Since the right member of (4.5) is absolutely and uniformly convergent for 
I 2 — 1 I ^ 1 for arbitrary X, it may be expressed as 

(4.6) + + 

Since the radius of convergence of the right member of (4.6) is some value 
such that I 2 — 1 I < 7?2 > 1, it may be expressed as a power series about 2 = 0, or 




(4.7) viz) 




^ , T ^ r ^ I 

^ ax + 2 ! + 




1 + X* + '^ + 


■). 


Recalling now the definition of <p{z) as given in (3.1), there is obtained by equat¬ 
ing coefficients of like powers of z in (3.1) and (4.7) 

/ 

(4.8) 


p(r) = + L. ^ + ■. 


\e "^X” 

7 r! * 


Since it may l>e readily shown that 


(4.9) 

where 

and 


—X yT 

_= (-1)”A” 

dX" r! ^ r! 






X’’“' 


r! (r — 1)! 


^ X _ ^n- 1 ^ X 


r! 


rl 


(r-1)! 


we may also write (4.8) as 


e~^X" 


—X \ T 

. e X 


L:k . 3 C ^ X*^ 


(4.10) P(rt - - |=A-'--. + 


6 . Necessity. Assume that the function p(r) of (2.1), for arbitrary X, is 
given by the absolutely convergent series 


(5.1) 


p(r) = ^ 


, , . a Lj a" 


■) 9 ' 


Since c /r\ is continuous with respect to X, there follows, where 2 is a complex 
variable and I 2 I ^ 1 


2 ^ 6 “'X" 


^z'pir) = Z) , 

r.O ri.0 r 1 


z'e-^\' 


r! 


^ 2! ax»;a rl ~ ^ 


(5.2) 


= e"'-’ 


” + Li(z — 1 ) ^ ~ 1 )* + • ’ • ^ 


1 + M,(z - 1) + (2 - 1)» + ff (a - D’ + 
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where 

(5.3) = L„ + «L„ X + L„-. X» + ... + X". 

From (5.2) it follows that 

(5.4) = M(n) 

where /i(«) is as defined in (3.3). Since (5.1) becomes for r = 0 , X = 0 

(5.5) 1 — M(l) 4- 2\ ~ g'j M(3) + • • • 

the assumed absolute convergence implies that 

(5.G) 1 + I ixa) I + 21 I I 31 I I ‘ ‘ I I I + • * • 

converges. 


6 . Remarks. ObrechkofT [11 ] shows that his result includes those of Pollaczek- 
Geiringer [ 12 ], Szegei [ 12 ] (p. 110 ) and Jacob [5]. His theorem states that if 
the function p(r), (r = 0 , 1 , 2 , • • •), satisfies the following conditions 

( 0 . 1 ) |p(r) I 

r«l 


is convei^ent for each finite numlicr A , and 


( 6 . 2 ) 


_(^)" f.|;Kr)J 

(// + I)!£i r 


(e-^X7r!)"^ 


tends toward zero as n increases indefinitely then p(r) may be expressed in a 
convergent Charlier Type B series, 
llsj^ensky [18] shows that if 


(0.3) 


12 z' p(.r) 

r-0 


has a radius of convergence R > 2 then p{r) may be expressed in a convergent 
Charlier Type B series. 

Schmidt [ 10 ] shows that a necessary and sufficient condition for the convergence 
is that the function ip(z) defined as in (3.1) (he does not explicitly impose the 
condition ( 2 . 1 ) on p(r)) be regular inside the two circles | z | < 1 and | z — 11 < 1 
and with all its derivatives is continuous on the peripheries also. In the case 
that p(r) ^ 0 , the condition (2.5) is stronger, in fact in this case Schmidt [16] 
shows that a necessary and sufficient condition is that 

lim p(r) 2 V* = 0 



578 


S. KULLBACK 


for all integral A; S 0. If p(r) ^ 0, then Uspensky's condition is only just 
enough stronger than Schmidt's to keep it from being sufficient. 

If (6.1) is satisfied, or if (6.3) is satisfied then (3.1) is absolutely convergent 
for I 2 I g 2. Therefore, the point 2 = 2 is contained in the circle of convergence 
of (3.2) or (3.4) which implies that 

1 + I M(l) I + I M(2) I + • • • + “1 I I + • • • 

converges. 

It is deemed worthy of special mention to point out, as both Schmidt and 
Uspensky have done, the striking fact that the necessary and sufficient condition 
for the validity of (2.4) is independent of X. This arbitrariness of X enables us 
to dispose of it so as to obtain better convergence. Indeed if we set X = /i(i) 
then as is evident from (4.2) Li = 0. 


7. Special cases. It is of interest to note that (4.8) is the Taylor expansion 
if p(r) = (r = 0, 1, 2, • • • ), for then (4.2) becomes 

(7.1) = (m - X)" 


since for the Poisson Exponential Distribution r V/r!, (r = 0, 1, 2, • • •)» 
M(n) = m” ^d (4.8) is then 


(7.2) 


e ^ tx 


r! 


« dc"'X’' , (m - X)* d'e""X’‘ , 

-FT + dx-H- + -21- av^-TT + 


If p(x) is finite, that is if p{r) = 0 for r > n + 1 then = 0 for A; > n -f !• 
Thus, for a finite function the condition (2.5) is satisfied. 


8. Factorial moments. For functions p{r), (r = 0,1,2, • • •), satisfying (2.5), 
there may be derived from (3.1) and (3.4) the relation 

(8.1) r!p(r) = /lo - jii(Hi) + ^|M(r+2) - + ••• , (r = 0, 1, 2, •• •), 

since each side is derived respectively from (3.1) and (3.4). It should 

be noted that for X = 0 (4.5) leads to (8.1) rather than (4.8) so that (8.1) may 
be considered as the Charlier Type B series for X = 0. The result (8.1) was 
derived for finite fimctions by Steffensen [17]. (Also compare Kaplansky [8]). 
This may also be expressed symbolically by 

(8.2) p(r) = (r = 0, 1, 2, • • •), 

where after expansion is to be replaced by M(n) . It is of interest to note the 
relation between the symbolic expression for p{r) as a Poisson Exponential in 

(8.2) and the series (4.8)', for (4.8) may be expressed symbolically as 

PW = (X + LYM 

(8.3) 

since = f{x + o) and the relations (4.2), (4.3), (4.4). 
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9. Illustrations. Consider the function 

(9.1) p(r) = (r = 0, 1, 2, • • •). 

For this function 

(9.2) v>(*) = Zz'pCr) = 1/(2 - 2) 

r«0 

and 

(9.3) V>‘“’(1) = M(») = n! 

SO that (2.5) becomes 

(9.4) 1 + 1 + 14 .... 

which does not converge. (It may be of interest to note that for this case 
( 8 . 1 ) yields 

(9.5) p(0) = 1-1 + 1~1 + 1 --.... 

The series on the right in (9.5) is not convergent but is summable Cj to |. For 
the latter see for example R. P. Agnew, [19].) In this case the first several co¬ 
efficients of (4.8) are for X = 1, 

= 0 , = .5000, I? = .3333, y = .3750 

(9.0) 

= .3667, = .3681, ^ = .3679, 

51 (>1 /! 

Let us now consider the function 

(9.7) p(0) = I, vir) = Vz', (r = 1, 2, •••). 

For this function 

(9.8) <p{z) = = I + 

r-O 6 — Z 

and 

(9.9) ^‘"’(1) = M(») = (n = 1, 2 , •• • ), 

so that (2.5) becomes 


(9.10) 
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which converges. For this case (8.1) yields 


(9.11) 

s 

II 

1 

i\l,(Z\l /3\1 

,2/2 V2/2» V2/2« 

• = i 

- ( 1 )^ 

V2/2* ^ 2!\2/2» 

• = i 

etc. 





In this case, the first several coefficients of (4.8) are for X == 0.75 

L, = 0, ^ = .093750, ^ = .046876, = .019043 

( 9 . 12 ) 

^ = .010840, I? = .005173, ^ = .002622, • • • . 


Let us now consider the function (suggested by Prof. C. Wexler) 


(9.13) p(0) = ^ 


> 1 ( 1 )'. 


p(r) = (—I)' 

) 

For this function 

(9.14) 23p(r) = 1, X 1 p(r) 1 = 5 


1 , 2 , •••). 


(9.15) 


viz) = E^Xr) = 5/(3 + 2z) 

r-0 


(9.16) ,.‘”>(1) = M(,) = (- 1)"^ (2/6)" 

In this case (2.6) becomes 


(9.17) 




■+(?)■+ 


which converges and (8.1) yields 


(9.18) 


p(0) = 1+ J + 


(S)'-(Si 


+ 


p(l) = -2/5 - 2!(2/5)^ - (2/5)* 


= 5/3 


5 

3 


2 

3 


etCr 

Note that for this case (6.1) or (6.3) are not satisfied. Using X = 1, it is 
found that 

(9.19) Lx = -1.4, ^ = 1.06, I* = -.6906, = .2779, ••• . 
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NOTES 

This section is devoted to brief research expository articles on methodology 
and other short items, 

ON SMALL-SAMPLE ESTIMATION 

By George W. Brown 
Iowa State College 

1. Summaxy. This paper discusses some of the concepts underlying small 
sample estimation and reexamines, in particular, the current notions on “un¬ 
biased’* estimation. Alternatives to the usual unbiased property are examined 
with respect to invariance under simultaneous one-to-one transformation of 
parameter and estimate; one of these alternatives, closely related to the maxi¬ 
mum likelihood method, seems to be new. The property of being unbiased in 
the likelihood sense is essentially equivalent to the statement that the estimate 
is a maximum likelihood estimate based on some distribution derived by inte¬ 
gration from the original sampling distribution, by virtue of a “hereditary” 
property of maximum likelihood estimation. 

An exposition of maximum likelihood estimation is given in terms of optimum 
pairwise selection with equal weights, providing a type of rationale for small 
sample estimation by maximum likelihood, 

2. Introduction. In large sample theory of estimation the problems are 
generally formulated in terms of a random variable x = {xi, X 2 y • • • , Xn) and a 
product distribution with, say, a density g{x\e) = f{xi\e)f{x 2 \ 6 ) •••/(xn|0 ) 
where n is permitted to increase without limit. For small sample theory it is 
sufficient to consider an arbitrary distribution, not necessarily of product form, 
depending on a parameter $. For convenience we will assume a distribution 
density of fixed form g{x\d)y where x is in Euclidean w-space and d in Euclidean 
&-space, k <n. Granting at the outset that a complete rationale for estimation 
must be based on considerations like those of Wald [4,1939] dealing with specified 
risk functions, it is still a difficult process, in practice, to specify the risk functions 
and solve the ensuing mathematics problems. It may still be to the point, then, 
to consider general properties that estimates might be required to have in order 
to be considered “acceptable”, or perhaps even “optimum”, over a class of 
“acceptable” estimates. 

In large-sample theory the situation is fairly simple. Consistent estimates 
have the property that the estimate converges in probability to the true param¬ 
eter value. “Best” or “optimum” estimates are defined in terms of the order 
of convergence, or asymptotic vari^ce. All reasonable definitions of “optimum” 
become asymptotically equivalent, since they all measure essentially the rate of 
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convergence, so that one might ask for least variance, or least expected absolute 
deviation, or least expected A;th power, without affecting the optimum estimate, 
in general. Moreover, the consistency property and the optimum properties 
are in general invariant under simultaneous one-to-one transformation of the 
parameter and its estimate, i.e., the square of an asymptotically optimum esti¬ 
mate of a will be an asymptotically optimum estimate of <r*. Finally, a general 
estimation method, the method of maximum likelihood, leads to optimum esti¬ 
mates in large samples. 

In small samples, on the other hand, the search for corresponding criteria has 
led to the investigation of best “unbiased” estimates, and the like, where few, 
if any, of the definitions discussed possess an invariance property imder simul¬ 
taneous one-to-one transformation of the parameter and its estimate. 

3. Unbiased estimation. To ensure, in small-sample estimation, that an 
estimate bears some relation to the parameter it is estimating, it has become the 
custom to require that an estimate be unbiased^ which means that the expected 
value of the estimate agrees with the parameter value. This condition was sug¬ 
gested by the consistency property which is required in large-sample estimation. 
It ensures, moreover, that the average of a large number of independent estimates 
made on the same basis will provide a consistent estimate, in the large sample 
sense. While this consistency property of the average may at times be conveni¬ 
ent in practical situations, the fact remains that the problem of estimation from 
a number of such observations is a different estimation problem, the “best” 
solution to which need not be the average of the “best” solutions of the original 
problem corresponding to estimation of d from a single observation on x, where 
X has a density g{x\d). More to the point, however, is the objection that an 
unbiased estimate of a parameter does not in general transform into an unbiased 
estimate when both estimate and parameter are subjected to the same one-to-one 
transformation. Moreover, one can easily construct situations for which the 
only acceptable unbiased estimates are clearly inferior from almost any point 
of view, to estimates which are biased (Girshick, Mosteller and Savage, [1,1946], 
and Halmos [2,1946]). 

It may be of interest to consider a few reasonable alternatives to the lack of 
bias requirement, which seem to accomplish as much as the conventional defini¬ 
tion and which, in addition, have an invariance under one-to-one transformation 
of the parameter and estimate. To avoid confusion, let us attach the qualifying 
prefix “mean” to the usual unbiased property, so that an estimate will be said 
to be mean-unbiased if its expected value agrees with the parameter value. 

Consider as one alternative the following property. An estimate of a one- 
dimensional parameter 6 will be said to be median-unbiased, if for fixed $, the 
median of the distribution of the estimate is at the value 0, i.e., the estimate 
underestimates just as often as it overestimates. This requirement seems for 
most purposes to accomplish as much as the mean-unbiased requirement and 
has the additional property that it is invariant under one-to-one transformation. 
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A different alternative requirement which is invariant under transformations 
is suggested by the definition of unbiased tests ofjjsignificance (Neyman and 
Pearson [3,1936]). Let us say that an estimate is likelihood-unbiased if \ B') < 
h{B I 0), where the estimate d has probability density A(d ||0). Infother words, an 
estimation method is likelihood-unbiased if estimates in the neighborhood of a 
given parameter value B would occur more frequently when the true value is 
itself B than when it differs from B. On intuitive grounds this seems to be an 
acceptable kind of requirement, applicable to a very general class of estimation 
problems. It is evident that the assumption of a density plays no important 
role here; the situation is analogous to the maximum likelihood situation. The 
property itself is invariant under simultaneous one-to-one transformations of 
parameter and estimate for the same reason that maximum likelihood estimates 
are invariant under such transformations, in fact one can readily see that the 
likelihood-unbiased condition is equivalent to requiring that d have such a 
distribution, as a function of Bj that the maximum likelihood estimate of B 
based on d will be actually equal to L The obvious implication of this fact is 
that if a function <t)(x) is given (possibly a sufficient statistic for B) then there is 
an essentially unique likelihood-unbiased estimate 6 based on obtained by 
finding the maximum likelihood estimate of 6 in the distribution of as a function 
of 

As an example, consider the estimation of from a sample of n observations 
from a normal distribution. Let be the usual sum of squares, where S^/a^ 
is distributed like x on n — 1 degrees of freedom. Then the only likelihood— 
unbiased estimate of based on is S^/in — 1). In this case I) is 

also mean-unbiased, a fact which is normally quoted as justification for the 
division by w — 1. Curiously enough, it is customary to estimate c by 
\/sy{n — 1), even though this is a biased estimate of o-, according to the usual 
notion of ‘‘unbiased’^ referred to here as “mean-unbiased”. On the other hand, 
\/S^/(n — 1) is a perfectly good likelihood-unbiased estimate of <r, by virtue 
of the invariance under transformations. It might be pointed out, in passing, 
that the estimate S^/{n — 1) does not have minimum mean square about 
but that the optimum divisor for minimizing the mean square error about 
isn + 1. 

The fact that a likelihood-unbiased estimate is the maximum likelihood esti¬ 
mate based on the distribution of the estimate itself suggest further examination 
of maximum likelihood estimates. If we define a simple estimate as one which 
completely determines a probability distribution for x, then we have as a theorem, 
the following: 

A simple maximum likelihood estimate d{x) is likelihood-unbiased. What this 
means is essentially that maximum-likelihood is “hereditary”, i.e. if d(x) maxi¬ 
mizes g{x I 0) in a space of n dimensions, and & has a derived density h{d | B) 
in a space oik <n dimensions, then ^ = d maximizes h(jB | B ). The proof follows 
readily from the fact that h{d | B) is obtained by integration of g(x j B) over all 
X such that Hx) — d. 
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The example of estimating a-*, quoted above, shows that the word “simple” 
cannot be omitted from the statement above. For example, the simple estimate 
in the parent distribution is the joint estimate (x, ^/n) of (m, a) and in fact the 
joint estimate is likelihood-unbiased. On the other hand, is not a simple 
maximum likelihood estimate, and we observe that ^/n is not likelihood-un¬ 
biased. >S*/(n — 1) is a simple maximum likelihood estimate of <r* based on 
the distribution of itself, so that iS*/(n — 1) is, as a result, likelihood unbiased. 

One can exhibit situations in which the conventional mean-unbiased property 
is very unnatural, while the likelihood-unbiased property may be quite natural. 
Consider, for example, the case where a is to be estimated by use of a x*“dis- 
tributed with n — 1 degrees of freedom, but subject to the condition o-* > <to , 
where qI is known in advance. Then the estimate cr® = max [S^/{n — 1), <t\] is 
certainly biased according to conventional definitions, but is nevertheless, likeli¬ 
hood unbiased. To get a mean-unbiased estimate when tr* is near to <ro is im¬ 
possible except by admitting estimates less than , which is clearly foolish if it is 
known that cr* > <to . 

It may be of interest to include a brief discussion of maximum likelihood esti¬ 
mation in terms of pairwise selection of alternatives, providing a sort of optimum 
property for maximum likelihood estimation in small samples, in addition to the 
likelihood-unbiased property. Consider a choice to be made between only two 
alternative values of B, say Bq and Bi , by dividing the sample space into two 
regions iSo and , such that is accepted when x falls in So and B\ is accepted 
when X falls in Si . Then 

-f Poo(5i) = Pe,(So) + Po,(Si) - 1. 

PeiiSo) is the probability of making the error of accepting Bq when B = Bi and 
1 ~ 's the probability of making the error of accepting Bi when 0 = • 

If the two errors are weighted equally, it is evident that a “best” test will choose 
aSo so as to minimize Ptfi(So) + 1 — Pdo{So). It is well known that So will 
minimize the indicated quantity if So consists of all points x such that g{x j Bo) > 
g{x I Bi). Thus we may speak of the region So defined by g(x | Bq) > g(x | 0i) 
as an optimum equal risk acceptance region for Bo against Bi, Now if we transfer 
(3ur attention to the general estimation problem we see that the maximum 
likelihood estimate d{x) is that value of B which would be accepted by the op¬ 
timum equal risk acceptance procedure against all other ^*s. 
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A NOTE ON REGRESSION ANALYSIS 

By Abraham Wald 
Columbia University 

1. Introduction. In regression analysis a set of variables y, Xi, • • • , Xp 
is considered where y is called the dependent variable and Xi, • • • , Xp are the 
independent variables. T^et ya denote the ath observation on y and the 
ath observation on x,, (i = 1, • • • , p; a = 1, • • • , AT). The observations x,« 
are treated as given constants, while the observations pi, • • • ^ys are regarded 
as chance variables. The following two assumptions are usually made concern¬ 
ing the joint distribution of the variates , * • * , 2/jv : 

(a) The variates pi, • • • , are normally and independently distributed with 
a common unknown variance <r*. 

(b) The expected value of ya is equal to ftxia + • * * + where ft , • • • , 
ft are unknown constants. 

In some problems it seems reasonable to assume that the regression coefficients 
jSi, • • • , jSp are not constants, but chance variables. This leads to a different 
probability model for regression analysis and the object of this note is to discuss 
certain aspects of this model. In what follows in this note we shall make the 
following assumptions concerning the joint distribution of the chance variables 
2/i) * ‘ ‘ I yN\ ft»* * *) ft • 

Assumption 1, For given values of , • • • , jSp the joint conditional prob¬ 
ability density function of i/i, • • • , 2/j^r is given by 

1 r 1 ^ 

(1«1) (2^)^/*^^ I ””2? ^ — • • • — ft Xpa) 

Assumption 2, The regression coefficients /3i, • • • , Pp are independently 
distributed. 

Assumption 3. The regression coefficients jSi, • • • , ft , (r < p), are normally 
distributed with zero means and a common variance <7'*. 

. cr'* 

The purpose of this note is to derive confidence limits for the ratio —. Such 

confidence limits have been derived by the author [1] for analysis of variance 
problems assuming that there are only main effects but no interactions. The 
regression problem treated in the present note is much more general and in¬ 
cludes all the analysis of variance problems with or. without interactions as 
special cases. 

It should be remarked that Assumptions 2 and 3 do not exclude the case where 
jSr+i, • • • , ft are constants. 

(t'* 

2. Derivation of confidence limits for the ratio —. Let 5i, • • • , &p be the 
sample estimates of j8i, • * • , jSp obtained by the method of least squares. We 
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shall denote the difference — jSi by €<, (t = 1, • • • , p). It is known that for 
given values of ft , • • • , j3p the conditional joint distribution of €i, • • • , €p is 
normal with zero means and variance-covariance matrix || || cr* where 

(2.1) lk.ill = l|a.7ir^ 

and 

N 

(2*2) aij = JCiaiJCja’) (tf j = 1, * * • , p). 

a-1 

Since the conditional distribution of ci, • • • , Cp does not dep)end on the values 
of /3i , • • • , i3p , the unconditional distribution of ei, • • * , Cp is the same as the 
conditional one, and the set of variates (jSi, • • • , pp) is independently dis¬ 
tributed of the set (ci , • • • , Cp). From this and Assumptions 2 and 3 it follows 
that bi , • • • , 6r have a joint normal distribution and that 


(2.3) 

and 

Ebi = 0, 

(f = 1, • • • , r) 

(2.4) 

Ebibj = ^c,v -b Sii ff*, 

i = 1, • • • , r) 

where 8,7 = 

0 for i ^ j and = 1 for i = j. 

.4 


We shall denote ^ by X and the elements of the inverse 
<r^ 

dij{\)y i.e., 

of II Cii + Sii\ II by 

(2.6) 

11 dii(X) 11 = 11 Cij -f SifK 11 \ 

a, 3 = 1, ••• ,r). 


Then the quadratic form 

(2.6) Q(\) = 

has the distribution with r degrees of freedom. 

It is known that for any given values of |Si, * • * , jSp, , • • • , 6p the quadratic 
form 

(2.7) Qa ~ ( 2 /a hiXia * • • i^p^pa) 

/T* 


has the x^ distribution with N — p degrees of freedom provided that the rank 
of the matrix || Xia || is p. Hence Qa and Q(X) are independently distributed 
and the ratio 


( 2 . 8 ) 


j, _N - p Q(X) 

^ -—r-^ 


has the /’’-distribution with r and N — p degrees of freedom. 
Let Fi and Ft be two values chosen so that 


(2.9) 


Prob. {Fi g F g Ft] = c 
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where c is a given positive constant less than 1. Then the set of all values X 
for which the inequality 

( 2 . 10 ) ^ ~ P ^ < Ft 

T (ala 

holds forms a confidence set for X with the confidence coefficient c. 

We shall now show that Q(X) is a monotonic function of X and, therefore, 
the confidence set determined by (2.10) is an interval. Let || (7*7 |1 , (t, j = 
1, • • • , r), be an orthogonal matrix and let 

( 2 . 11 ) = 

/-I 

It then follows from (2.3) and (2.4) that 


(2.12) 

Eibt) = 0 , 

(i = 1, • • • , r) 

and 



(2.13) 

Eibtb^) = (cf, + 

{i,j = 1 , ••• ,r) 

where 



(2.14) 

Cii = 25!^ gikOiiCki. 

1-1 ik-l 


Let 



(2.15) 

II d?y(X) II = 11 + 5<yX |r\ 

(.i.j = 1 , ••• , r) 

and put 

Q*(X) - iz2d?,•(X)6?5^ 

O'* 


It is easy to verify that Q*(X) is identically equal to Q(X). Hence, to prove 
the monotonicity of Q(X), it is sufficient to show that Q*(X) is a monotonic func¬ 
tion of X. Since no restrictions as to the choice of the orthogonal matrix H flfiy 1| 
are made, we shall choose it so that the matrix \\ctf\\ becomes diagonal, i.e., 
cfy = 0 for i 9 ^ j, (i, y = 1 , * • • , r). Then 

(2.16) 

d«(X) = 0 

for i 7 ^ j 


and 

(2.17) d?<(X)-;*-^. 

Cii + X 

Hence 

(2.18) Q(X) = Q*(X) = \i 

tr* i»i Cii + X 

is a monotonically decreasing fimction of X. The confidence set determined by 
(2.10) is, therefore, an interval. 
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The upper end point of the confidence interval is the root in X of the equation 
(2.19) 

r Qa 

and the lower end point is the root in X of the equation 


( 2 . 20 ) 


AT - p Qi\) ^ 
r Qa 


If equation (2.20) has no root, the lower end point of the confidence interval 
is put equal to zero. 
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ON THE SHAPE OF THE ANGULAR CASE OF CAUCHY’S 
DISTRIBUTION CURVES 

By Aurel Wintner 

The Johns Hopkins University 

1 . Let { be a linear random variable, that is, a random variable capable of 
values X represented by points of a line — <» <x < «, and suppose, for sim¬ 
plicity, that i has a density of probability, f{x). Then, subject to provisos of 
convergence, the series 

Fix) - /(x + n) 

nmm—90 

represents a periodic function, of period 1, having the following significance: 
F(x) is the density of probability of the angular random variable, say H, which 
is obtained if all the states 

f-2, {-1, f, f+1, i + 2, ... 

of the linear random variable are identified. 

In other words, if a circle of unit circumference rolls from — oo to oo on the 
{-line, then every point of the circumference collects the various densities of 
probability attached to congruent points of the f-line, and a state of E repre¬ 
sents a point of the circumference. For a detailed study of the mapping { S 
or/~>F, cf.[2]. 

According to Poisson’s summation formula, the Fourier constants of the 
periodic function F(fc) can be obtained by restricting u in g{u) to an equidistant 
sequence of discrete values, where g{u) denotes the Fourier transform of /(x); 
cf., e.g., [6], p. 78 or [9], pp. 477-478. 
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2. Consider, in particular, the case in which f(x) is the density of a S3rmmetric 
distribution which i s stable in Cauchy’s sense. The determination of the totality 
of these linear densities of probability is due to L4vy [6]. It was shown in [8] 
that every such f{x) = /(—x) is a decreasing function of | x j. As explained in 
[8], p. 70, this fact makes superfluous one of the axioms occurring in Gauss’ 
postulational approach to “errors of observation.” 

The purpose of the present note is the deduction of the angular analogue of 
the fact just quoted. The analogue states that, if f(x) is symmetric and stable, 
then the corresponding periodic F(x) is decreasing for 0 ^ a: ^ § (and so, for 
reasons of symmetry, is increasing for J ^ a; g 1). This is contained in the 
italicized statement of §4 below. 

In view of Poisson’s rule, quoted above, the periodic densities in question can 
be defined by certain Fourier scries representing generalizations of elliptic theta- 
series. From this point of view, not even the existence (i.e., the positivity) of 
.the periodic densities is obvious, if arbitrary values of the “precision constant” 
(denoted below by 5) are allowed. The difficulties involved are explained in §3. 

3. If q and X are positive constants the first of which is less than 1, then the 
(even, periodic) function 

(1) hix ; g) = 1 + 2 2 g"' cos nx, 

nmml 

where > 0, has derivatives of arbitrarily high order at every real x. It is 
regular-analytic at every real x if and only if X > 0 is replaced by X ^ 1, where 
the sign of equality holds if and only if the analytic continuation (from the a:-axis) 
is not an entire function. In fact, it is known that a Fourier series 
S(an cos nx + sin nx) is that of a function which is regular-analytic at every 
real x, and has the period 2 t, if and only if | a„ | -f | 6n | is majorized by a con¬ 
stant multiple of the nth power of a positive constant which is less than 1; 
and that the latter constant can be chosen arbitrarily small if and only if the 
analytic continuation does not lead to any singularity (at a z 7^ 00). 

Since the function (1) tends to 1 uniformly in a; as g —» +0, if X is fixed, there 
belongs to every X > 0 a positive g* = q*{\) having the property that 

(2) B\{x ; g) > 0 for 0 ^ a: < 2 t 

if 0 < g < g*(X). It is less obvious that, if g is sufficiently small with reference 
to X, say if 0 < g < g**(X), then 

(3) B^{x ; g) is decreasing for 0 ^ x ^ ir 

(hence, increasing for ir g x < 2ir). The existence of such a g**(X) < 00 for 
every X > 0 can be assured as follows: 

If 8n{x) denotes the nth partial sum of the Fourier series S(sin nx)ln^ then 
s„(x) is positive for 0 < x < tt (Gronwall, Jackson; for a short proof, cf. [4]). 
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Hence, a partial summation shows that the sum of a sine series, Xbn sin nx, 
must be positive for 0 < a; < if 

nhn ~ (n + l)5n+i > 0 and nK —> 0. 

Since the^first derivative of (1) (with respect to x) results by choosing 
hn = — 2715^" , it follows that (3) must be true if 

nY" - (n + > 0 

holds forn = 1, 2, • • • . But the last inequality is readily seen to be satisfied 
from n = 1 onward if, while X is fixed, q tends to 0. This proves that ^**(X) 
exists for every X > 0. 

4. From these deductions alone, it is quite unexpected that (the best values of) 
both q*{\) and q**i\) turn out to be independent of X when 

(4) 0 < X g 2, 

i.e., that (1) satisfies both (2) and (3) for 0 < q < 1, if (4) is assumed. This 
fact is of statistical significance, since, on the one hand, it is precisely the restric¬ 
tion (4) which is necessary and sufficient for the existence of Cauchy’s (s3nn- 
metric) “stable” distributions (cf. [6], pp. 254-263) and, on the other hand, 
the reduction (mod 27r) of the densities of these linear distributions leads to 
the functions (1) as angular densities (cf. [9], pp. 477-478); the numerical value 
of q{ < 1) being determined by the “precision” or “dispersion” of the resulting 
angular distributions. 

Under the necessary restriction (4), the linear analogue of g*(X) = 1 and of 
g**(X) = 1 was proved in [6], pp. 258-263 and in [8], pp. 71-77, respectively. 
It will remain undecided whether the restriction (4) is necessary in either of 
the angular cases. 

5. Suppose that X has a fixed value in the range (4). Then there exists a 
monotone function of <, say ax(0> for which 

exp (—w^) — / exp {—'(it) da\{t) 

Jo 

is an identity in u, where 0 < w < oo (cf. [1], p. 769, where further references 
will be found). Hence, a change of variables shows that 

g”' = f deed 1 log q 
Jo 

is an identity in q and n, where 0 < g < 1 and n = 0,1, 2, • • • (the integration 
variable is t). Consequently, from (1), 

0x(x;«) = f Oiix i q‘) dcti^d I log q 

JQ 
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where 0 < g < 1 and — « < a; < «. In fact, the legitimacy of the term-by- 
term integration is obvious from 0 < g < 1 and da\ ^ 0 (even though the inte¬ 
grals are improper). 

6 . Since ox is a non-decreasing function, it is clear from the last formula line 
that both (2) and (3) will be proved for 0 < g < 1 and for every X (satisfying (4)), 
if it is ascertained that both (2) and (3) hold for 0 < g < 1 when X = 2. But 
the case X = 2 of (1) is an elliptic theta-function, for which both properties in 
question (cf. the diagram in [3], p. 44) are known; a simple proof can be con¬ 
cluded from what, in Hecke^s terminology, is the Eulerian factorization of 
02(x ; g), as follows: 

According to Jacobi, the factorization of the case X = 2 of (1) is 
;g) = n (1 - g*")(l + 2?”-* cos x + 

Tl«l 

(cf. [7], pp. 64-65). Thus 

9j!(x ;g) = Cg n + «• 

flaal 

where 

C, = n (1 - g’") 

n«l 

and 

(5) P{x ;r) = 1 — 2r cos X + r*, (0 < r < 1), 

hence 

P(x ; r) > 0 (0 < r < 1). 

Since 0 < g < 1, this proves the case X = 2 of (2). Furthermore, logarithmic 
differentiation of the product representation of ; g) gives 

6 t(x ; g) = 02 {x ; g) X P'(x + r ; g’""‘)/P(x + »; g’"“‘), 

n—1 

where/' = df/dx; so that, by (5), 

P'(x + T; r) = —2r sin x. 

Since 0 < g < 1, the last.three formula lines and the case X ~ 2 of (2) imply that 

Oi(x;g) < 0 if 0 < X < T, 

as claimed by the case X » 2 of (3). 

This completes the proof of the italicized assertion. 
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a[note on the fundamental identity of sequential analysis 

By G. E. Albert 

U. S. Naval Ordnance Plant, Indianapolis 

1. Introduction. Let ja:,), (t = 1, 2, 3, • • •), be a sequence of real valued 
random variables identically distributed according to the cumulative distribution 
function F(s). Define the sums Zn — Zi + + * * • + Zn iov every positive 

integer N. Choose two positive constants a and b and define the random vari¬ 
able n as the smallest integer N for which one of the inequalities Zn a or 
Zir ^ —6 holds. The notations P{u j F) and E{u | F) will denote the probability 
of u and its expectation respectively assuming that F is the distribution of the Zi, 

Wald [ 1 ] has established the results contained in the following lemmas. 

Lemma 1. If the variance of F{z) is positive, Pin < oo | F) egmls one. 

Lemma 2. If there exists a positive number d such that P(e* < 1 — 5 | F) > 0 
and Pie* > 1 + 5 I F) > 0 and if the moment generating function <pit) = | F) 

pxists for aU real values of t, then <pit) has one and only one minimum at some finite 
value t ^ to. Moreover, fp"it) > 0 /or all real values of t 

It is the purpose of this note to establish the following extension of the validity 
of certain results given by Wald [ 1 ], [ 2 ]. 

Theorem.^ Under the conditions of Lemma 2 the identity 

( 1 ) E{e^^^l<pit)r\F\ = 1 

^Wald's results show (1) to be valid for all complex t in the domain over which | ^(01 ^ 1 
and the validity of the differentiation clause for all real t in that domain. The import¬ 
ance of the present extension arises from the fact that, if Eix | F) 0, then 0 < ^(0 < 1 
on a certain interval of the real axis. 
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is valid and may be differentiated with respect to t under the expectation sign any 
number of times for all real values of t. 

Proof. The notation to will be used consistently to denote the t value at which 
tp{t) has its minimum. 

The proof of the theorem follows Waldos methods quite closely and certain 
of the results given in [1] and [2] will be used here without discussion. 

Consider first the validity of (1). For an arbitrary positive integer N let Pn 
be the probability Pin ^ AT | F) and let Esiu | F) and E*iu | F) denote the 
conditional expectations of u subject to the respective conditioas n ^ N and 
n > N, Wald [1] has shovm that for any finite real value of t 

(2) PsEAe^^^l<p(t)r \F} + il^ P^n^piOr'^EUe^^^ \F] =1. 

Since lim PsEjf{ [^(<)r”exp(ZnO} is the left member of the identity (1), it suffices 
to demonstrate that 

(3) lim (I - \F\ =0 

JV-oo 

^or all real values of t. 

Since 1 — Ps tends to zero with increasing N and the expected value 
involved in (3) is bounded independently of N for any fixed the only source of 
difficulty in proving (3) lies in the fact that <pit) may be less than unity on an 
interval of the real axis. That difficulty is easily avoided by the following 
device. Define the function 

(4) G(x) = y(to)rj_\“>dFiz). 

Obviously Gix) is a distribution function whose moment generating function 
^(t) exists for all real L Its mean is zero and its variance is positive as will be 
seen from the equations Eix | G) = ^'{Q/^iU) and Eix^ | G) = It 

follows that ^(0 is never less than unity for real values of t 
Let Q denote the space of all , • • • , Zn and let n(n > W) be that subset of Q 
on which n > N. One has 

[ e^‘'*dFizi)---dF(z„) f 

jQ(n>N) __ «^Q(n>y) _ 

f e*"' dF(zi) ■ ■ ■ dF(z„) f dG(zi) • • • dG(z„) 

Jo Jq 

where s ^ t — U and Qir = P(n ^ AT 1 G). By Lemma 1, 1 — Qjv tends to 
zero as W is increased. Thus, since ^(s) ^ 1 for all real t and the expected value 
I (?} is bounded independently of N for a fixed t, the equation (3) holds 
for all real L 
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The differentiability clause of the theorem requires the following modification 
of a very powerful theorem due to Charles Stein [3]. 

Lemma 3. Under the conditions of Lemma 2, if the minimum <p{Q of fp(f) is 
less than unity, there exists a positive number U such that 

(5) E[exp [ntx — n log ^(<o)l | F} < oo. 

Proof. If G is the distribution of the , by Stein's theorem there exists a 
positive number U such that | G) is finite. 1^1 fi(n = N) denote the 

subset of Q on which n = N. Then 

P(n = IG) = f dO{zt) • • ^dGiz^,) 

- f e^^^^dF{z,y^^dV{zs) 

•'Q(n-N) 

^ P{n = AT I F) exp [minja/o, — 6?o} — N log ^(/o)l. 

It follows that 

F{exp [nti — n log ^(^o)] 1 F) ^ 1 G] exp [— minfafo, — hto]] 

and the lemma is proved. 

To continue with the theorem, Wald's proof [2] suffices for the case in which 
fpito) ^ 1. Attention w'ill be given only to the case <p{to) < 1. As pointed out in 
section 2 of [2], the differentiability clause of the theorem will be established if 
it can be shown that for any finite interval I of the real axis and any pair of 
integjsrs n and ri there exists a function , n) such that for all t in I 

one has 

(6) Dr,r,{Zn , u) ^ \ I 

and 

(7) F{Z)rif,(^n , n) 1 F} < 00 . 

On referring to Wald's proof and using the inequality —log ^(0 ^ —log ip{to) for 
all t in /, it is seen that there exists a constant C and a positive number U such 
that the function 

Dr,r,iZ„ , n) s 

satisfies (6) for all t in I. To establish (7) use the inequalities (2.4) and (2.6) 
in Wald [2] to obtain 

E\Drir% {Zn | w) | F} 

ct. Pin = N\F)mk)r’'En^ {6'*“ + «"**“ IF} 

AT-l 

^ exp'hlogn- nlog»>(<o)l|F}, 


( 8 ) 
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That (7) is indeed satisfied now follows from (5) and the finiteness of the function 
1(0 since for a large enough integer 3f one has 

E F(n = iV IF) exp [r, log AT - AT log 


^ E Pin = N\F) exp [Nh - N log vik)] < «>. 

Thus the expected value on the extreme right in (8) is finite. This completes 
the proof of the theorem. 
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A SIGNIFICANCE TEST AND ESTIMATION IN THE CASE OF 
EXPONENTIAL REGRESSION 

l^Y D. S. VlLLARS^ 

United Slates Rubber Company, Passaic, N, J. 

1. Introduction. The principal problem under consideration in this note 
may be described as follows. Consider a variate, z, whose distribution for a 
given value of a fixed variate, t, is: 

(1.1) /(2 I t) = 

where a, b, and k are real-valued parameters. The regression of 2 on i is exponen¬ 
tial, for it follows from (1.1) that the expected value of z, given ty is; 

(1.2) E(z\t) = a- 

On the basis of a random sample Oivfe , h ; Z 2 , ( 2 ; * " ; , /at) it is desired to 

test whether A; = 0 or 00 . The problem of “fitting*' a curve, 2 = a — 
to the sample (^. e. of estimating a, h, and k from the sample) will also be treated. 
As an illustration of how the statistical problems described above arise in 
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practice, let us consider a typical situation in industrial chemistry. Let the 
quantity, z, be a property of a latex and let the quantity, t, be time. Suppose, 
furthermore, that measurement of t is without error but that measurement of z 
is subject to error; let it be assumed that the observed value in a measurement of 
2 is a variate having a normal (Gaussian) distribution about the “true value,*' 
E{z). On basis of N independent measurements, 2i, 22 , • • • , of 2 at times, 
ti t U i • • ^ i iff, respectively, the experimenter may wish to test the hypothesis 
that A; = 0 or 00 . If this hypothesis is true the suspected exponential relation 
between z and t does not hold; in this case E(z) is a constant (a ~ 6, or o) and 
estimation of the constant from the data is quite straightforward. If the data 
conflict with the hypothesis that A; = 0 or 00 , the experimenter may wish to 
estimate the parameters, a, 6, and k (i. e., “fit” the curve, z = a -- he~^\ to the 
data). 

The problems considered in this note will be treated only for the case where N 
is an even integer (> 6) and the times < 1 , < 2 , • • * , ^j^r at which measurements of 
z are made are such that 

(1.3) ka — ka-\ = A, a constant, (a = 1, 2, • • • , n = N/2), 

The odd time intervals, <3 — , 4 — ^ 4 , etc. do not have to be equal. 

2. Test of the hypothesis that A; = 0 or <». The space, say Q, of admissible 
values of the parameters in (1.1) is: o-* > 0, — «> < a, 5, A; < +00. Under the 
null hypothesis the admissible values of the parameters lie in a subspace of 12, 
say CO, specified as follows: cr* > 0, ~ < a, 6 < + <», A: == 0, or 00 , 

Let yj = 22 a and Xj = Z 2 a -\, (a = 1, • • • , n = N/2), From (1.1) and (1.3) 
it follows that the n pairs x ,, y, are normally and independently distributed with 
common variance, <r^, that x, and yj are independent (j = 1, 2, • • • , n), and 
that 

(2.1) i/y = A + rnnj 

where vj = E{yj)^ ^.j = E{x,)y A = o(l — e~*^), and m = The space, 

12', of admissible values of the parameters in the joint distribution of Xj, yj, 
(j = 1, • • • , w), is: a > 0, Vj - h + myj , — 00 < /i < + 00, — 00 < iij.Vj < 
+ 00 ; 0 < m < <». The subspace of 12', say co', associated with the null hypoth¬ 
esis is: V > 0, yy = My == c, where c = o — 6 or a according as A; = 0 or 
In 12', the expected values of x and y lie on a line; in co' they lie in a single point. 
It is clear that by transforming the original sample 0]^(zi, k zn , t]f) to & 
sample 0n(:ci, ; • • • ; > ^n) we have reduced the original problem to the 

familiar problem of linear regression in which there is “error in both variates”. 

The slope of the “line of best fit” to the sample points {xi,yi; ••• ;Xn, Vn) 
is [Ij: 

(2.2) m = + V{S„ - S„y + 4SJ,J/2-S.. 
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where 

Sxx s ^ (Xj - 3cf 

1 

r 

Sxv = H (xj - x){ys - y) 

1 

Syy = Z iVi - Vf 
1 

n 

2 ^ Z Xj/n 
1 

n 

2/ = Z 2/j/» 

1 

(w is an estimate of m in (2.1)). Since m = (where k and A are real), it is 
intuitively clear that when m is non-positive the sample 0^ does not conflict 
with the null hypothesis. The null hypothesis can be tested by means of the 
statistic [2, 144] 

qN pf __ Sxx + 2mSxv + wi* Syy 

^ ^ 7ri^Sxx-2mSxy + Syy' 

The null hypothesis is rejected if m is positive and F' is large. Percentage points 
of the distribution of F' are given in [2, 146] forn = 3 (1) 15 (5) 30, 40, 60, 120 
and for significance levels, 0.001, .01, .05, .10, and .20. These significance 
levels, however, were computed for use in cases where the sign of m was irrele¬ 
vant. It happens that to test the null hypothesis under consideration in this 
problem at a significance level a we should use a critical value of F' (given in 
[2]) corresponding to a significance level 2 a. The reason for this is that when 
the null hypothesis is true the quantities m and F' arc independent and the 
probability that m is positive is J—thus the chance of rejecting the null hypoth¬ 
esis is \{2a) = a, 

3. Estimation of a, 5, and k. If the data do not support the hypothesis that 
A; = 0 or 00 , the experimenter may wish to estimate a, 6, and /b. General alter¬ 
native methods of estimating these parameters will now be considered. 

(1) Estimate o, h, and k from Oat by the method of least squares; i.e., solve 
the simultaneous equations dS/da == 0, dS/dh = 0, and dS/dk = 0 for a, 6, 
and A;, where 

(3.1) -S = Z ( 2 .- - a + 

The value of k obtained by this method of estimation will not in general be the 
same as that computable from ni in (2.2) and used for the significance testing. 

(2) Estimate k by means of (2.2) and the relation m = then substitute 
this estimate into S oi (3.1) and estimate a and 6 by means of least squares. 
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(3) Estimate A; as in (2) and choose, as an estimate of a, the intercept of the 
“line of best fit” for On . Then substitute these estimates of o and k into (3.1) 
and estimate h by means of least squares. In this case the estimate of h comes 
out to be: 

(3.2) S = i: -zd/t, e-^‘ 

where d and Ic are the estimates of a and k. 

If the values, • • • , are such that U+i - f,- = A, (i = 1,2, • • • , AT - 1), 
the following estimation procedure might be used. 

(4) Let 

Vi ® 

{i = 1, 2,..., N - 1), 

Xj = Zi 

and treat the (AT - 1) pairs of values (xi, ; • • • ; Xs^i , Vn-^i) as a sample of 

size (N — 1). Using this sample, estimate k, a, and 6 in a manner similar to 
that in (2) or (3). It should be noted that this sample is not a random sample 
owing to the dependence among the {N — 1) elements. 

The procedure in alternative (1) is very laborious and time-consuming. The 
procedure in (2) and (3) can be carried out quickly and easily. In (1) the 
method of least squares yields the same results as would be obtained from appli¬ 
cation of the method of maximum likelihood. Examples of estimation by proce¬ 
dures (3) and (4) are given in the next section. 

4. Example. The accompanying table lists experimentally observed values 
of a property of a latex obtained at biweekly intervals. Using the first, third, 
etc.j quantities as Xj and the remaining ones as yj , the sums of squares and prod¬ 
ucts of deviations are found to be: 

= .035510 X = 0.9195 

= .025645 

&yy = .023414 y = .9365. 

Substituting these values in equation (2.2) and computing the other constants 
from equation (2.1) we get: m = 0.791596, o = 1.0009, and k == 0.1168. The 
E' ratio is (2.3) 17.03. Entering Table I of [2], we find that for eight point pairs 
a value of E' = 16.5 may be expected only one time in one hundred. On ex¬ 
cluding the possibility of negative values of m, this corresponds to the 0.5% 
signihcance level. The exponential relationship is thus concluded to be highly 
significant. 

Evaluation of h by equation (3.2), method 3, gives 0.2560, if all 16 values are 
used. The equation cdculated from the data is thus: 

(4.1) 2 = 1.0009 - 0.2560 
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The alternative procedure, method 4, would be to use all the Zi points for the 
estimation of a and k. This leads to the following values of the computation 
quantities: 

S„ = E ** - *?• = 0.052374; £ = 0.9223 

16 

^xy = 2 ~ .036924 

16 

Syy ^ H x] - x\ = .035436; y = .9381. 

t-ii 

Note that the difference Syy — Sxx used in the formula for m cancels out all inter¬ 
vening squares between the first and last. 

Syv “ Sxx = Xl - X\t, 

TABLE I 


i 

weeks 

1 

i 

weeks 

U 1 

t 

weeks 

Si 

1 

t 

weeks 

Si 

1 

.776 

9 

.939 

17 

.942 

25 

.955 

3 

.852 

11 


19 

.938 

27 

.993 

5 


13 


21 

.979 

29 

.985 

7 

.869 

15 

.948 

23 

.975 

31 

1.013 


However, the data excluded thereby are in effect included in the new Sxy . 

The final values obtained by the fourth procedure are: m = 0.796596, a = 
1.0000, and k = 0.1137. The writer does not know whether the peculiar trans¬ 
ference of data from Syy — Sxx to Sxy characteristic of procedure 4 improves the 
accuracy of the fit or hurts it. It is his personal preference to use procedure 3. 

6. Acknowledgement. The writer wishes to acknowledge with thanks his 
gratitude to Drs. T. W. Anderson, Jr. and David F. Votaw, Jr. for many sug¬ 
gestions and discussions concerning this problem and for much help in clarifying 
the presentation of the concepts. 
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ON THE POWER EFFICIENCY OF A t-TEST FORMED 
BY PAIRING SAMPLE VALUES 

By John E. Walsh 
Princeton University 

1. Introduction. Consider two equal sized samples, one from a normal popu¬ 
lation with mean /x and the other from a normal population with mean v. Let 

, • • • , be the sample values from the population with mean m and 2 / 1 , • • •, j/n 
the values from the population with mean v. If the two populations have the 
same variance and the two samples are independent, the most powerful tests 
for comparing y and v using these samples (one-sided and symmetrical two- 
sided) are based on the statistic 

^ - y - (m — v)]\/n{n — 1 ) 

tj •= - - -. 

if + 2 (^i - 9f 

which has a Student ^-distribution with 2 n — 2 degrees of freedom. Tests based 
on U also have the desirable property of being invariant under permutation of 
the data in each sample. 

Sometimes, however, it is useful to combine the sample values in the form 

Zi = {xi - Vi), (t = 1 , • • • , n). 

Examples: 

(a) . When the samples are independent but it is not known that the two popu¬ 
lations have the same variance (Behrens-Fisher problem). 

(b) . When there may be correlation between Xi and (i = 1 , ••• , n), 

this correlation being the same for each value of i (i.e. Xi is independent of yj 
if t 3 ^ j while each pair a;,*, , (t = 1 , • • • , n), has the same normal bivariate 

distribution). 

In both (a) and (b) the are independently normally distributed with the 
same variance and mean y — v. 

The Student ^test for comparing y and v using the z,* is based on the statistic 
— y)]\/n(n — 1 ) _ [$ — g — {y — v)]\/n{n — 1 ) 

which has a Student ^-distribution with n — 1 degrees of freedom. These tests 
are not invariant under permutation of the data in each sample. 

If it is true that all the sample values are independently distributed with the 
same variance efficiency will be lost by using the test based on h instead of 
the most powerful test based on U • The purpose of this note is to determine the 
power efficiency of the tests based on ii as compared with the corresponding 
tests based on ^ for this case. 




TABLE I 

Power Function Values for the k and t 2 Tests 
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Consideration is limited to one-sided tests, which is not a serious limitation 
since any two-sided test can be considered as a combination of two one-sided 
tests. Table II contains approximate power efficiencies of one-sided tests for 
n > 4 at the significance levels a = .05, .025, .01. 

It is found that the efficiency of the ti test increases with the sample size but 
is high even for small size samples. 

2. Outline of computations. The method of obtaining power efficiencies 
used here will be that outlined in [1]. Essentially this consists in computing the 
power function for the test based on h and then adjusting the sample size for 
the corresponding test based on U until its power fimction is approximately the 
same as for the h test. The ratio of the sample size (perhaps fractional) of the 
adjusted U test to that of the h test is called the power efficiency of the h test. 
Intuitively this efficiency measures the fraction of the total available information 
which is being used when the h test is applied (since the test is most powerful) 

TABLE II 


Approximate Power Efficiencies for Given n and a 



4 

5 

6 . 

7 

8 

9 

10 

15 

25 

00 

.05 

82.5% 

85% 

87% 

88.5% 

90% 

91% 

92% 

95.5% 

98% 

100% 

.025 

77%* 

80%* 

82.5% 

84.5% 

86.5% 

88.5% 

90% 

93% 

96% 

100% 

,01 

73% 

75.5% 

78% 

80% 

82% 

84% 

85.5% 

90% 

94.5% 

100% 


* These values were obtained by comparison with the corresponding values for 
a = .05 and .01. 


It is easily seen from symmetry that a one-sided h test of p < v has the same 
power efficiency as the corresponding one-sided h test of p > v. Thus it is 
sufficient to consider the one-sided tests of p > v. 

The power function is found as a function of the parameter 6, where 



Most of the approximate power efficiencies were determined by using the 
normal approximation given in [2] to compute the power function values. This 
approximation was used for fractional values of n. Table I contains the results 
of these computations for one-sided tests of p > v, 

Exac]^ values of the power function for integral values of n and a = .05, .01 
can be found frpm the tables in [3]. A comparison of the power function values 
obtained from the normal approximation with these exact values shows that, 
forn < 6, a = .01 and n < 4, a = .05, .025, the approximation underestimates 
the true values for small 5 and overestimates for large 5. Although this combina¬ 
tion of underestimation and overestimation tends to cancel out in the determina- 
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tion of power efficiencies, so that little error in power efficiencies would be 
expected if the approximation were used forn = 6, a = .01 orn = 4, a = .05, 
the efficiencies given in Table II forn = 4, a = .05 and n = 4, 6, a = .01 were 
obtained from the exact values by graphical interpolation and cross-interpolation. 

Power efficiencies were not considered for n <4 because of the difficulties 
of interpolation and the inexactness of the normal approximation in this range. 

For n = 00 , and U both have a normal distribution with zero mean and unit 
variance. Thus the power efficiency is 100% at all significance levels for 
this case. 

These computations furnish approximate power efficiencies for n = 6, 8, 10, 
15, 25, 00 at a = .05, .025, .01, and for w = 4 at a = .05 and .01. The other 
approximate power efficiencies listed in Table TI were obtained by graphical 
interpolation from these values. 

The results of this note can be roughly summarized for n < 15 by stating 
that of the 2w sample values 

(i) . approximately 1.6 values are lost at the 5% significance level, 

(ii) . approximately 2.1 values are lost at the 2.5% significance level, 

(iii) . approximately 2.8 values are lost at the 1% significance level, if the 
tests based on h are used instead of the corresponding tests based on h . Exami¬ 
nation of Table I shows that the number of sample values lost decreases as n 
increases for n > 15. 
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NOTE ON THE LIAPOUNOFF INEQUALITYFOR ABSOLUTE MOMENTS 

By Maurice H. Belz 
The University of Melbourne 

For a variate x measured from the mean of the population, the absolute 
moment of order r is defined by 

Vr = f \x\' dF{z), 

J—co 

where F{x) is the cumulative distribution function. Treating r as continuous, 
we have 

^ I * r log. I * I dF{x), 

the integral on the right existing if exists. 
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Write y = log* Vr* Then we have 


xr ^ 1 * r log, I a: I dF{x), 

xj ^ I a: r dF{x)- I ® T logi I ® I dF{x) - |£^ | a: I' log, j» | dF{x) | 


^ 0, by Schwarz^s inequality. 



It follows that the function y is convex (or exceptionally a straight line), and, 
on referring to the figure, it appears that 

(1) MQ ^ MQ' 

for all chords PR, If the abscissae of the points L, M, N are c, 6, a, respectively, 
where c ^ 6 ^ a, the inequality (1) leads at once to the relation 

log* Vb S - log* Pe H-log* Pa . 

Q'^C 0“”C 


Hence 

o~o ^ a—*6 h—’O 
Vh ^ Ve Va 7 

which is the usual form of the Liapounoff Inequality. 

REMARK ON THE NOTE “A GENERALIZATION OF 
WARING’S FORMULA” 

By T. N. E. Grevillb 

U. S, Public Health Service 

Before submitting for publication the note “A generalization of Waring’s 
formula,” Annals of Math, StaL^ Vol. 16 (1944), pp. 218-219 the author made a 
diligent effort to ascertain, through correspondence with mathematicians and 
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actuaries both in this country and abroad, whether the generalized formula in 
question had been previously published, and none of the authorities communi¬ 
cated with knew of its prior publication. However, it has now come to his 
attention that the formula was published in essentially the same form by Hermite 
in the article ‘‘Sur la formule d*interpolation de Lagrange”, Journal fiir die 
Reine und Angewandte Mathematik (^'Crelle’s Journal”), Vol. 84 (1878), 
pp. 70-79. 
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1. Estimation of Parameters in Tnmcated Pearson Frequency Distributions. 

A. C. Cohen, University of Georgia. 

Given a truncated univariate Pearson frequency distribution, parameters of the com¬ 
plete distribution are required. Karl Pearson and Alice Lee, {Biometrikat Vol. 6 (1915), 
pp. 59-69) and R. A. Fisher, {Introduction to Mathematical Tahlea^ Vol. 1, British Assn. 
Adv. Sci., 1931, pp. xxvi-xxxv), obtained solutions of the truncated normal distribution 
with a single tail missing. The present paper presents three general methods of solution 
applicable to any of the Pearson distributions. The first utilizes moments of a higher order 
than are required to characterize corresponding complete distributions. The order of 
the highest moment required is increased by one for each missing tail. The second method, 
applicable when only a single tail is missing, utilizes the terminal ordinate at the point of 
truncation and moments of the same order as required to characterize the complete dis¬ 
tribution. The terminal ordinate is evaluated by successive approximations. The third 
method utilizes only the first two moments, but requires that the given distribution be 
further truncated and that moments be computed both before and after the additional 
truncations. This latter method can also be applied to complete distributions to avoid 
direct computation of third and fourth order moments. 

2. Distribution of a Root of Determinantal Equation. D. N. Nanda, University 
of North Carolina. 

The joint distribution of the roots of a determinantal equation was given by P. L. Hsu 
in 1939 and the distribution of any one of the roots was studied by S. N. Roy. The present 
paper, however, gives a different method of working out the distribution of any root, 
specified by its place in a monotonic arrangement. This method enables us to express the 
distribution of a root of a certain determinantal equation in terms of a linear combination 
of products of incomplete beta integrals and in terms of the distribution of a root of lower- 
order determinantal equations. 

3. The Power of Certain Non-Parametiic Tests of Independence. Wassily 
Hoeffding, University of North Carolina. 

Several tests of independence have been proposed which are based on statistics depending 
only on the ranks of the sample values. Under the hypothesis Ho of independence the 
distribution of such statistics does not depend on the form of the parent distribution. 
Two of these statistics, Spearman’s rank correlation coefficient and lindeberg-Kendall’s 
statistic based on the number of inversions in the permutation of the ranks, are shown to 
be asymptotically normally distributed in samples from any population (the limiting nor¬ 
mal distribution being singular in certain degenerate cases). The asymptotic distribution 
of these coefficients reveals that the corresponding tests of independence are inconsistent 
(in the sense that the probability of rejecting Ho does not necessarily tend to 1 if Ho is not 
true), and at least one of them is biased in the limit. It can be shown that at least for some 
sample sizes and some sizes of the critical region there do not exist unbiased tests of inde¬ 
pendence based on ranks. But there do exist rank tests of independence which are con¬ 
sistent, and hence unbiased in the limit. Examples of such tests are given. 

4. Some Significance Tests for the Mean Using the Sample Range and Midrange. 

John E. Walsh, Princeton University. 
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Consider a sample of size n, (2 < n < 10), drawn from a normal population with mean/i. 
Let Xn be the largest value and Xi the smallest value of the sample. Significance tests are 
developed to compare m with a given hypothetical value no by use of the sample. These 
significance tests are based on the quantity D » + ^») no]/{Xn — xi) [(sample 

midrange) — (hypothetical mean)!/(sample range). One-sided and symmetrical tests are 
considered. Values of Da such that Pr(D > 2>a | m “ /wo) ■* a are computed for a » .06, .026, 
.01, .006. These values of Da can be used to obtain one-sided tests at the .06, .026, .01, .006 
significance levels and symmetrical tests at the .10, .06, .02, .0]« significance levels. Effi¬ 
ciencies are computed for one-sided tests at the .06,and .01 significance levels. The effi¬ 
ciency is at least 90% forn < 6 at the .06 significance level and for n < Sat the .01 level. 
The range-midrange test can be applied without computation through the use of an easily 
constructed graph. The application of a test requires only the plotting of the sample 
point (xi , Xa) on this graph. 

5. Testing Compound Symmetry in a Normal Multivariate Distribution. David 
F. VoTAW, Jr., Princeton University. 

Let F{X) be the d.f. of a (-order vector variate X(( ;> 3). Suppose the components of X 
are divided into mutually exclusive and exhaustive subsets. F{X) is said to be compound 
symmetric, for the given division of its variates into subsets, if it is invariant over all per¬ 
mutations of its variates within these subsets. F(X) is completely symmetric if the invari¬ 
ance holds over all permutations of its variates. If F{X) is normal and compound sym- 
metric, then within each subset of variates the means are equal, the variances are equal 
and the covariances are equal, and betweeft any two subsets of variates the covariances 
are equal. Testing hypotheses of compound or complete symmetry in a normal F{X) 
is of interest, for example, in studying psychological examinations and in medical research. 

In this paper likelihood ratio criteria are developed for testing various hypotheses 
involving compound symmetry in regard to a normal distribution and to k normal dis¬ 
tributions {k '^2). Given that the corresponding null hypothesis is true, the moments 
of each criterion are obtained explicitly and the distribution of each criterion is identified 
as the product of independent beta variates (in the case of a single normal distribution, 
the distributions are given explicitly for ( — 3,4, and 6 for certain divisions of the variates 
into subsets). In a previous paper Wilks has given results on a very thorough study of 
the sampling theory of likelihood ratio criteria for various hypotheses involving complete 
symmetry in regard to a normal distribution. 

6. Effects of Non-Normality at High Significance Levels. Harold Hotell¬ 
ing, University of North Carolina. 

The effects of non-normality in the underlying population on the probabilities of sig^ 
nificance by customary statistical tests are not well understood, in spite of numerous 
attacks, both mathematical and experimental, on the problem. Chung’s recent proof that 
.the distribution of the Student ratio ( has in samples from an arbitrary population a dis¬ 
tribution approaching normality for large samples tends to confirm the common idea that 
non-normality makes little difference if only the sample is fairly large, but this holds 
only for a fixed range of values of t while the sample number N increases. The tail areas 
beyond a deviation which increases with N in certain ways often behave quite differently 
than in sampling from a normal population. If p is the probability that | (| > (o in sam¬ 
ples of N from a normal population and p' is the corresponding probability for another 

population, it is shown that lim infinite or may take any 

finite value, even when the non-normal distribution involved is of simple and realistic 
continuous forms. The conditions that this limit be unity are concerned only with the 
Mhotdders of the population histogram, and have nothing to do with its moments or ita 
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behavior at infinity or at its mean. This suggests that caution should be used in applying 
familiar tests with high significance levels; that further calculations should be directed 
toward making this caution quantitatively definite; and that the use of sample moments 
or cumulants cannot lead to the most appropriate criterion of non-normality for this 
purpose. 

7. On the Problem of Similar Regions. E. L. Lehmann, University of Cali¬ 
fornia, Berkeley, and Henry Scheff^, University of California, Los Angeles. 

If X = (Xi, ••• , Xn) is a set of random variables with a joint probability density 
depending on a set of parameters 0 ^ (^i, • • * i 0m), and if T — (Ti , • ■ * , T«) is a set of 
sufficient statistics for then Neyman (Phil. Tram. Roy. Soc. Ijondon^ Vol. 236 (1937), 
pp. 333-380) has proved that a region w in the space of X is similar with respect to e if it 
has the following structure: The intersections w{Jt) of w with the surfaces T ^ i have the 
property that the conditional probability of the sample point X falling into w given that 
T « i does not depend on 1. In the present paper a necessary and sufficient condition is 
found for the regions with the above structure to be the only similar regions. This con¬ 
dition is shown to be satisfied for a certain class K of probability densities which contains 
as special cases all densities for which the totality of similar regions has been previously 
determined. In particular the partial differential equations which Neyman {Annals of 
Math. Stat., Vol. 12 (1941), pp. 46-76) assumed were satisfied in his solution of the problem 
of similar regions arc solved and it is shown that any density satisfying these equations 
belongs to the above class K. 

8. Fourth Degree Exponential Function. L. A. Aroian and Marguerite 
Darkow, Hunter College. 

It is shown that the fourth degree exponential function is supported by the Bernoulli 
probability function and the hypergeometric probability function as well as being the 
function for which the method of moments is the best method according to the criterion of 
maximum likelihood. In the general situation six moments, at most, are needed. The 
function is classified into two general groups depending on symmetry or asymmetry and 
each case is divided again into unimodal and bimodal distributions. Examples show that 
the function is very successful in graduating the main Pearson types and the Gram-Charlier 
Type A frequency function. Various generalizations of the exponential function are 
indicated. In addition to its wide generality, the greatest practical advantage of the new 
system is the simplicity of the numerical calculations. 


9. A General Weak Limit Theorem for Independent Distributions. P. L. Hsu, 
University of North Carolina. (Read by title.) 


For every positive integer n let therei be n distribution functions Fni{x)f Fntix), 
••• , Fnn{x). Assume that limn-*«o Maxi^yg.ll - F»,(x) + Fnj{-x)] « 0. Let F{x) 


be the convolution Fni{x)*Fnt{x)* *Fnn{x). Let ^(<) 


/•+* 

mil -f J l6‘** — 


1 - itx/(l + x*)l (1 + X*) /x* dO (x), with 0{x) T and (7( « ) - (7(- « )< « . Let F(x) be the 
(infinitely divisible) distribution law having exp ^(t) as its characteristic function. 
In order to have lim»-,.F*(x) «• F(x) at every continuity point of F(x), it is necessary 
and sufficient that the following relations hold at every x > 0 such that ±:x are continuity 
points of 0(y): 


(I) limn-*oo 



|»|>» 



|v|>* 


(d + y*)/y*) dO(y), 
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ai) f ^ dF„i(y> -( f y dFn,(yu\ - f (1 + »•) dO (y), 

J-1 {J \v\>z V l»|<» / J J lv|<» 

(III) lim„-*ooZ^ f ydFni(y)^m+ f y dG(y) - f (1/y) dO(y), 
j-i J \y\<z J |yl<* J |v|<» 

10. On the Maximiun Partial Sums of Sequences of Independent Random 
Variables. K. L. Chung, Princeton University. 

The asymptotic behavior of the maximum partial sums of a sequence of independent 
random variables is studied in this paper. Two groups of new limit theorems are estab* 
lished under general conditions. The first group deals with theorems of the weak type. 
The limiting distribution of the maximum partial sums is obtained with an estimate of 
the remainder, thus improving a recent result of Erdos and Kac. Another estimate is 
obtained for a different domain of variation, which plays an essential role in the sequel. 
These results correspond to the sharper forms of the central limit theorem. In the second 
group, theorems of the strong type are obtained, giving precise lower bounds (in the sense 
of probability) for the maximum partial sums. These results form the exact counterpart 
to the general form of the law of the iterated logarithm, due to Feller, which give the pre¬ 
cise upper bounds. A summary of the main results and methods has appeared in Proc. 
Nat. Acad, of Set., Vol. 33 (1947), pp. 132-136. 

11. Some Results on the Distribution of Quadratic Forms From Gaussian 
Stochastic Processes. (Preliminary report). Herman Rubin, Cowles 
Commission. 

If one considers the estimation of the parameters of a Gaussian stochastic process 
whose elements are continuous functions from the functional values over a finite interval, 
one often finds that certain parameters can be estimated exactly, and certain parameters 
can not. This result often depends on the distribution of quadratic functionals whose 
arguments arc elements of the stochastic process under consideration. In this paper, it 
is shown that the elements of a certain class of quadratic functionals have distributions 
concentrated at a point, and that the elements of a different class do not; in this latter case, 
the characteristic function is computed. 

12. Some Significance Tests for the Median which are Valid under Very General 
Conditions. (Preliminary Report) John E. Walsh, Princeton University. 
(Read by title.) 

Consider n independent values drawn from populations necessarily satisfying only: 1) 
Each population has a unique median. 2) The median has the same value ^ for each popu¬ 
lation. 3) Each population is symmetrical. 4) Each population is continuous. (It 
is to be emphasized that no two of the values are necessarily drawn from the same popula¬ 
tion.) Significance tests are derived for <p on the basis of l)-4). These significance tests 
are based on order statistics of certain combinations of order statistics, each combination 
being either a single order statistic of the n values or one-half the sum of two order statistics. 
The tests are invariant under permutation of the n values and reasonably efficient if the 
values represent a sample from a normal population. The significance levels are of the 
form r/2», (r « 1, • • • , 2** — 1) . Each value of r can be obtained for some one-sided signifi¬ 
cance test. Thus any significance level can be closely approximated if n is large. The 
major disadvantage of these tests is the limited number of suitable significance levels avail¬ 
able for small values of n. This disadvantage is partially eliminated by the development of 
tests which have a specified significance level if the values are a sample from a normal 
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population and a significance level bounded near this specified value if only l)-4) necessarily 
hold. Results based on l)-4) are applied to several well known statistical problems: 
Tests are obtained for the mean on the basis of a large number of independent values from 
populations having the mean but little else in common. Also generalized results are ob¬ 
tained for the Behrens-Fisher problem, quality control, slippage tests, the sign test and 
cases where some of the n values are dependent. 

13. Loss of Information in f-tests with Unbalanced Samples. (Preliminary 
Report) John E. Walsh, Princeton University. (Read by title.) 

Consider two normal populations, the first with mean ai and variance <r}, the second with 
mean 02 and variance aj, while a/ai has a known value C. If the hypothesis ai — oa is to 
be tested by a i-test (one-sided or symmetrical) using ni sample values from the first popu¬ 
lation and n 2 values from the second population (ni + n 2 = n, fixed), it is shown that this 
experiment is most powerful when rii/rii = <ri/<r 2 (integer considerations neglected). The 
f-tests satisfying this condition will be referred to as balanced t-tesis. Thus information 
will be lost by not using a balanced experiment. A quantitative measure of the information 
lost by using given values of ni and is determined by the total sample size m, (wi -f ma = 
m), of the balanced t-tost (same significance level) which has approximately the same power. 
Then n — m sample values are wasted by using (ni , n^) rather than (mi , m 2 ), i.e. only 
100m/n% of the information obtainable per observation is used by (ni , na). A sym¬ 
metrical t-test with significance level 2a has the same value of m as a one-sided f-test with 
significance level a. For one-sided f-tests with significance level a ; m = -i- \/r* — 8A), 

where R - 2 + A + KV2, A = (C + UHl - K* /2(n - 2)llCVni + iMl-b and Ka is the 
standardized normal deviate exceeded with probability a. This approximation to m is 
valid for m ^ 6 if a *= .05, m > 6 if a « .025, m > 7 if a «* .01, m > 8 if a == .006. (A 
fractional value of m represents an interpolated measure of the sample size of the equivalent 
balanced experiment.) 

14. Some Theorems on the BemouUian Multiplicative Process. T. E. Harris 
Princeton University. (Read by title.) 

A single entity may have j dcscendcnts with probability P,-, (; «= 0, 1, 2, • • •)• Each 
first generation entity has then the same procreative probabilities, etc. Let 

/(«) * po + PiS + • * • 

If Zn is the number of entities in the nth generation, it is known that P{zn « 3 ) is given by 
the coefficient of in the nth iterate/!/ • • • (/)] = fn{s). Let Ezi « x, 1 < a; < 00 . Con¬ 
ditions are given insuring that as n —> <» the cumulative distribution of the variate «»/«* 
approaches a limit-function which is absolutely continuous except for a possible single 
jump. Let g{u) be the corresponding frequency function. If/(8) is a polynomial of degree 
A;,letg « logx^/(log« A; — 1). Otherwise, g « 1. Then g(u) • exp[is, is not] summable 
(0, 00 ) according as € is [negative, positive]. Behavior of g{u) near n « 0 is also considered. 
Special cases are considered where g(w) *» constant • m a positive integer. Max- 

mum likelihood estimates for the parameters po, Pi i * * - > and x are obtained as functions 
of n successive values , 22 , • • • , Zn . Consistency, in a certain sense, is proved. A 
specialized method is given for finding the moment-generating function of the variate 
the smallest value of n such that z» 0. 



NEWS AND NOTICES 


Readers are invited to submit to the Secretary of the Institute news items of interest 

Personal Items 

Dr. George E. Albert has been appointed to an associate professorship at the 
University of Tennessee. 

Dr. T. W. Anderson, Jr, has been promoted to an assistant professorship in 
the Department of Mathematical Statistics at Columbia University. He is on 
leave the first half of the 1947-48 academic year at the Institute of Actuarial 
Mathematics and Mathematical Statistics, Stockholm University as a Guggen¬ 
heim Fellow. During the second half of the academic year he will be at Cam¬ 
bridge University. 

Associate Professor Max Astrachan has been promoted to a full professorship 
at Antioch College, Yellow Springs, Ohio. 

Associate Professor T. A. Bancroft, who has been at the University of Georgia, 
Athens, Georgia, is now with the Statistical Laboratory, Alabama Polytechnic 
Institute, Auburn, Alabama. 

Dr. M. S. Bartlett of Cambridge University has been appointed as Professor 
of Mathematical Statistics at the University of Manchester, Manchester, 
England. The position is a newly created one. Professor Bartlett indicates 
that this position is believed to be the first official professorship in mathematical 
statistics in England. 

Professor M. A. Brumbaugh has accepted a position with the Bristol Labora¬ 
tories Inc., Syracuse 1, New York. 

Dr. Donald A. Darling has been appointed Research Associate at Cornell 
University. 

Professor D. B. DeLury of the Virginia Polytechnic Institute has accepted a 
position with the Ontario Research Foundation, 43 Queens Park, Toronto 6, 
Canada. 

Professor Abel Gauthier of the University of Montreal has been appointed 
Head of the Institute of Mathematics and Assistant-Secretary of the Faculty of 
Science at that institution. 

Dr. Casper Goffman, former assistant professor in the Mathematics Depart¬ 
ment, University of Kentucky, is now in the Mathematics Department, Univer¬ 
sity of Oklahoma, Norman, Oklahoma. 

Mr. Philip Hardy has returned to the General Electric Company at Warren, 
Ohio after serving at Wright Field. 

Dr. Carl F. Kossack, who has been with the Navy Department in Washington, 
D. C. as an Air Intelligence Specialist, has accepted an associate professorship in 
the Department of Mathematics at Purdue University. 

Mr. Frank Jones Massey, Jr. is now teaching in the Department of Mathe¬ 
matics, University of Maryland, College Park, Maryland. 
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Dr. William Burton Michael, who has been Lecturer in Mathematics, Psy¬ 
chology and Educational Psychology at the University of Southern California, 
has now accepted an assistant professorship in the Department of Psychology, 
Princeton University. He is also a member of the Research Department, 
College Entrance Examination Board at Princeton. 

Mr. Bernard Ostle, a former teaching assistant. School of Business Adminis¬ 
tration, University of Minnesota, is now at Iowa State College, Ames, Iowa. 

Mr. Maurice H. Quenouille, who was formerly with the Rothamsted Experi¬ 
mental Station, Harpendon, Herts, England, has accepted the position of 
Lecturer in Statistics, Marischal College, University of Aberdeen, Scotland. 

Dr. James A. Rafferty left the Department of Pathology, University of 
Rochester in June and has been appointed Chief of the Department of Statistics, 
Air University, School of Aviation Medicine, Randolph Pield, Texas. 

Miss Mary Ann Savas has accepted a position with General Motors, Detroit, 
Michigan. 

Professor George J. Stigler, formerly with Brown University, is now teaching 
in the Department of Economics, Columbia University, New York, New York. 

Professor E. L. Welker has resigned an associate professorship in mathematics 
at the University of Illinois to become Associate in Mathematics in the Bureau of 
Medical Economic Research of the American Medical Association. 

Mr. Sol M. Wezelman, who completed his master's degree in actuarial science 
at the University of Michigan in June, has accepted a position as Assistant 
Actuary in the North Dakota State Department of Insurance, Bismarck. 

Dr. Bertram Yood has received his doctorate at Yale and is now on the staff 
at Cornell University. 

Mr. Earl K. Yost, Jr. has accepted a position with the General Electric Co. at 
the Hanford Engineering Project, Richland, Washington. 

Professor James G. Smith, of Princeton University, died at Princeton on 
November 28,1940. 


Beginning with the October issue, the quarterly journal Mathematical Tables 
and Other jVids to Computation will publish a new feature section, “Automatic 
Computing Machinery," designed to disseminate information and news on 
research and development in the field of high-speed automatic calculating 
machinery. Material should fall under the general headings of Bibhography, 
Technical Developments, Discussion (including correspondence), and News. 
Contributions to this section are invited and should be addressed to Dr. E. W. 
Cannon, Head of the Mathematics Group, Machine Development Laboratory, 
National Bureau of Standards, Washington, D. C. 


Institute of Numerical Analysis Established 
Plans have been completed for the establishment of one of the newest units of 
the National Bureau of Standards—the Institute of Numerical Arialysis—^at the 
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University of California at Los Angeles, according to an announcement by Dr. 
Edward U. Condon, Director of the Bureau. 

One of the giant high-speed electronic computing machines, now under devel¬ 
opment by the Bureau of Standards, will be installed at the Institute when 
completed. Design specifications call for high memory capacity and auto¬ 
matically sequenced mathematical operations from start to finish at speeds 
attainable only with electronic equipment. 

The Institute has two primary functions. The first is research in applied 
mathematics aimed at developing methods of analysis which will extend the use 
of the high-speed electronic computers. The second is to act as a service group 
for Western industries, research institutions, and government agencies. The 
service function will include not only the use of the machines for problem solving 
but also assistance in the formulation of problems in applied mathematics of the 
more complex and novel types. Service operations are to be initiated immedi¬ 
ately, using the latest types of commercially available computing equipment. 

The decision to locate the Institute at the University of California at Los 
Angeles was made after a nation-wide survey by the National Bureau of Stand¬ 
ards. Centers in the East and Middle West were considered as well as the Far 
West, but Los Angeles, it was decided, offered the widest range of possibilities 
for an Institute of Numerical Analysis. Concentration of aircraft industries and 
the presence of several major scientific institutions were critical in the choice of 
Los Angeles. 


Election of Fellows 

The Board of Directors announced at the Yale Meeting the election of the 
following members as Fellows of the Institute: Theodore W. Anderson, Jr., 
Alexander C. Aitken, David II. Blackwell, Georges Darmois, Ragnar Frisch, 
Robert C. Geaiy, Frederick Mosteller, Gerhard Tintner, Charles P. Winsor and 
John Wishart. 


New Members 


The following persons have been elected to membership in the Institute 
{June 1 to Augnst 29, 1947) 

Baldwin, Helen Mildred, B.S. (Cornell) Research Associate in Statistics, Atomic Energy 
Project, 215 Avenue C, Rochester 6, N. Y, 

Blunk, Paul M., A.B. Teaching asst, and grad, student, Univ. of Calif., Box 6S2, Fair 
Oaks, Calif. 

Bowden, George Edwin, B.S. (Duke) Teaching asst., Math. Dept., White Hall, Cornell 
Univ., Ithaca, N. Y. * 

Bradley, Ralph Allan, M.A. (CJuoen^s Univ.) Grad, student, Univ. North Carolina, Well- 
inglon, Ontario, Canada, 

Burton, Kenneth John, Head of Statistics Section, British Employers’ (Confederation, 16 
Rutherwyke Close, Ewell, Surrey, England. 

Carlson, Phillip G., Jr., A.M. (Columbia) 148 Cornell Street, Roslindale SI, Mass. 
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Carol, Bernard, M.S.E. ((^olunihia) Ctracluate student at Columbia IJniv., 16 West 96th 
Street, N. Y. 

Clark, Sidney B., B.A. ((icorge Wash. Univ.) Statistician, Public Hoads Administration, 
ms Porter St., N.W., Wash., 8, D. C. 

Danielson, Theresa, M.A. (Univ. of Ill.) Mathematician at Hmokluiven National Lal)ora- 
tory, S61B Cambridge Ave., New York 63, N. Y. 

Dineen, Russell D. F., B.A. (Univ. of Delaware) Graduate student at Univ. of Delaware, 
ISIS French Street, Wilmington, Delaware. 

Diver, M. L., M.K. (Purdue) Consulting Engineer, P.O. Box 1016, San Antonio 6, Texas. 

Erasmus, Josias C., M.S.E. (Univ. of Stellenbasch, South Africa) Research Officer, 
Grootfontein ('ollcge of Agriculture, Middelburg, C-P, South Africa. 

Gottlieb, Morris J., Ph.D. (Wash. Univ., St. Ix^uis) Member of the Institute for Advanced 
Study, Washington University, St. Ixuiis, Mo. 

Greenwood, Joseph Arthur, A.H. (Harvard) Student at Ilarvafd University, 66 Oxford 
St., Cambridge 38, Mass. 

Gysbers, Jack C., M.A. (Univ. of (^alif.) Teaching asst.. Dept, of Math., Univ. of Calif., 
2029 Berkeley Way, Berkeley 4, Calif. 

Haskind, Mina, B S. (Bn>oklyn College) Student at Brooklyn College, 763 Eastern Park¬ 
way. Brooklyn 13, New York. 

Hauser, Dr. Philip M., Ph.D. (Univ. of ('huago) University of Chicago, Chicago 37, Ill. 

Hoyt, Cyril J., Ph.D. (l^niv. of Minn.) Research Associate, Dept, of Education, Univer¬ 
sity of (^hicago, Chicago, Ill. 

Kem, Enrique Roberto, First Assistant, Institute of Biometry, Univ. of Buenos Aires, 
Ilivadavia 8854, Buenos Aires, Argentina. 

Mark, Abraham M., Ph.D. (Cornell) Mathematics Department, Univ. of Wisconsin, 
Madison, Wisconsin. 

Moss, George G., II, B.A. (St. John^s College, Annapolis) Actuarial Statistician, Metro¬ 
politan Life, 2771 Morris Ave., N. Y. 68, N. Y. 

Phillips, Bernard E., A.M. (Columbia) Box 147, Cathedral Station, New York 26, N. Y. 

Radvanyi, Laszlo, Ph.D. (Univ. of Heidelberg) Professor of Economics, National Univ. of 
Mexico, Donato Gverra 1, desp. 207, Mexico, D. F. 

Richardson, John M., Ph.D. (Cornell) Member of Technical Staff, Bell Telephone Lab¬ 
oratories, Inc., Murray Hill, New Jersey. 

Royston, Robert W., M.S. (Univ. of Mich.) Asst. Prof., Math. Dept., Wash. & Ix»e Univ., 
117 W. Wa^shington St., Lexington, Virginia. 

Savas, Mary A., A.B. (Univ. of Mich.) Student at Univ. of Mich., 624 E. Second St., Mon¬ 
roe, Mich. 

Shepard, David H., A.B. (Univ. of Mich.) Research Analyst, Anny Security Agency, 
606 Randolph Street, Falls Church, Virginia. 

Throdahl, Monte C., B.S. (Iowa State College) Research Chemist in Chaige of Rubber 
Lab., Monsanto Chemical Co., Nitro, West Virginia. 

Uchytil, Jan, Doctor of Science, Chief of Production Control Dep. in Central Federation 
of Czech. Industry, Praha II, Prikopy 14, Czech. 

Vergara, Jose, Doctor of Engineering, (Madrid) Professor of Economics, Madrid, Chief 
of the Bureau of Statistics, Dept. of Agric., Madrid. i262 S. Blackstone Ave., Chicago 
37, Illinois. 

Wei, Dcung-shu, Ph.D. (Univ. of Iowa) Prof, and Head of Math. Dept., St. John's Univ., 
Shanghai, 129 East 10th St., New York 3, N. Y. 

Wolfson, Jacob, B.A. (New York College) Statistician, Social Security Adm., 846 Bruns¬ 
wick Road, Essex, Maryland. 



REPORT ON THE NEW HAVEN MEETING OF THE INSTITUTE 

The Tenth Summer Meeting of the Institute of Mathematical Statistics was 
held at Yale University, New Haven, Connecticut, Tuesday, September 2 
through Thursday, September 4, 1947. The meeting was held in conjunction 
with the summer meetings of the American Mathematical Society and the 
Mathematical Association of America. The following 150 members of the 
Institute attended the meeting: 

C. B. Allendoerfer, 11. L. Anderson, H. E. Arnold, L. A. Aroian, II.M. Bacon, J. L. Barnes, 
W. D. Baton, R. E. Bechhofer, A. A. Bennett, Joseph Berkson, D. 11. Blackwell, C. I. Bliss, 
Colin Blyth, Jr., A. E. Brandt, G. M. Brown, 11. H. Brown, O. P. Bruno, P. T. Bruyere, 
Mrs. P. T. Bruyere, J. II. Bushey, B. H. Camp, G. C. Campbell, Uttam Cliand, K. L. Chung, 
W. G. Cochran, A. C. Cohen, Jr., E. P. C'olcman, T. F. Cope, G. M. Cox, C. C. Craig, E. L. 
Crow, H. B. Curry, G. B. Dantzig, M. D. Darkow, B. B. Day, Bernard Dimsdale, C. E. 
Dieulefait, C. W. Dunnett, Churchill Eisenhart, L. li. Elveback, M. W. Eudey, U. P. Evans, 
William Feller, C. D. Ferris, M. M. Flood, R. M. Foster, II. A. Freeman, J. E. Freund, H. P. 
Geiringer, M. J. Gottlieb, J. Arthur Greenwood, Evelyn Groosman, F. E. Grubbs, II. T. 
Guard, P. R. Halmos, Max llalperin, M. H. Hansen, B. I. Hart, Mina Haskind, Wassily 
HoeUding, R. H. Hoskins, Harold Hotelling, A. S. Householder, Jaroslav Janko, Irving 
Kaplansky, Leo Katz, Oscar Kempthorne, E. M. Kennedy, W. L. Kichline, C. J. Kirchen, 
L. F. Knudsen, II. S. Konijn, C. F. Kossack, Jack l4iderman, H. G. Landau, E. L. Lehmann, 
R. A. Leibler, Walter Leighton, Jr., F. C. Leone, Joseph I^v, Howard Levene, Julius Leib- 
lein, Arthur Linder, S. B. Littauer, E. D. l^owry, H. F. A'lacNeish, P. J. McCarthy, John 
Mandel, H. B. Mann, Sophie Marcuse, F. J. Massey, Margaret Merrell, E. B. Mode, M. E. 
Moore, Frederick Mosteller, D. N. Nanda, P. M. Neurath, M. C. Neurdeiiburg, G. E. Noe¬ 
ther, M. L. Norden, H. W. Norton, P. S. Olmstead, A. L. O^Toole, E. R. Ott, T. E. Oxtoby, 
fkiward Paulson, M. P. Peisakoff, G. B. Price, J. A. Rafferty, L. J. Reed, C. J. Rees, P. R. 
Rider, John Riordan, H. E. Robbins, Milton da Silva Rodrigues, A. C. Rosander, Ernest 
Rubin, Herman Rubin, Frank Saidcl, M. M. Sandomire, Arthur Sard, Max Sasuly, F. E. 
Safterthwaite, E. D. Schell, Jack Sherman, Rosedith Sitgreaves, Andrew Sobezyk, Milton 
Sobel, Herbert Solomon, Mortimer Spiegelman, Arthur Stein, Henry Teicher, R. M. Thrall, 
Gerhard Tintner, M. N. Torrey, J. W. Tukey, D. F. Votaw, Jr., Abraham Wald, H. M. 
Walker, J. E. Walsh, R. M. Walter, J. H. Watkins, Dzung-shu Wei, E. S. Weiss, S. S. Wilks, 
C. P. Winsor, H. O. Wold, Jacob Wolfowitz, C. A. Wright, Bertram Yood. 

The Tuesday afternoon session was devoted to a symposium on 2 x 2 tables 
with Professor Lowell J, Reed of Johns Hopkins University serving as chairman. 
Addresses were given on Tests of Significance by Dr. ChurchiU Eisenhart, Na¬ 
tional Bureau of Standards; Estimation by Dr. Charles P. Winsor, Johns Hopkins 
University and Non-Standard Cases by Dr. Joseph Berkson, Mayo Clinic. 
Discussants were Mr. William F. Taylor, Dr. Frederick Mosteller, Professor 
David H. Blackwell and Professor John W. Tukey. The attendance was 
approximately 130. 

The first Wednesday inoming session was devoted to contributed papers. 
Professor John W. Tukey of Princeton University presided. The attendance 
was approximately 85. The following three papers were presented: 

1. Estimation of Parameters in Truncated Pearson Frequency Distributions, 

Professor A. C. Cohen, University of Georgia. 
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2. Distribution of a Root of a Deterrninanial Equation. 

Mr. D. N. Nanda, University of North Carolina. 

3. The Power of Certain Non^Parametric Tests of Independence. 

Dr. Wassily Hoeffding, University of North Carolina. 

The second Wednesday morning session was held with Professor Will Feller, 
President of the Institute, presiding. Professor R. A. Fisher, University of 
Cambridge, gave the address under the title The Fitting of Gene Frequencies to 
Data for Genotypes. The attendance was approximately 160. 

The membership business meeting of the Institute was held at 9:15, Thursday 
morning, in 102 Chittenden Hall with President Feller presiding. The attend¬ 
ance was approximately 55. It was voted to make certain changes in the 
By-Laws and in particular to raise the due to $7.00 per year. (An exception is 
made for those living outside the Western Hemisphere.)*' Morris Hansen, 
Chairman of the Committee on Planning and Development, initiated a lively 
discussion with reference to desirable changes in the Constitution. 

On Thursday morning at 10:30, with President Feller presiding. Professor A. 
Wald of Columbia University presented the Henry Lewis Rietz Lecture on 
Sequential Estimation and Multi-Decisions. The attendance was approximately 
150. 

A joint session with the American Mathematical Society was held early 
Thursday afternoon at which Professor S. S. Wilks of Princeton University gave 
a lecture on Sampling Theory of Order Statistics. Professor Harold Hotelling of 
the University of North Carolina was the presiding officer. The attendance was 
approximately 300. 

This session was followed by another joint session with the American Mathe¬ 
matical Society which was devoted to contributed papers. Professor John W. 
Tukey presided at this session and the attendance was approximately 115. The 
following seven papers were presented; 

1. Some Significance Tests for the Mean Using the Sample Range and Midrange. 

Mr. John Walsh, Princeton University. 

2. Testing Compound Symmetry in a Normal Multivariate Distribution. 

Dr. David F. Votaw, Jr., Princeton University. 

3. Effects of Non-Normality at High Significance Levels. Professor Harold Hotelling, 
University of North Carolina. 

4. On the Problem of Similar Regions. 

Dr. Erich L. Lehmann, University of California, Berkeley and Professor Henry 
Soheife, University of California at Los Angeles. 

6. The Fourth Degree Exponential Function. 

Dr. Leo A. Aroian and Professor Marguerite Darkow, Hunter College. 

6. On the Maximum Partial Sums of Sequences of Independent Distributions, 

Dr. K. L. Chung, Princeton University. 

7. Some Results on the Distribution of Quadratic Forms from Gaussian Stochastic 
Processes, 

Mr. Herman Rubin, Cowles Commission. 

The following four papers were presented by title: 

8. A General Weak Limit Theorem for Independent Distributions. 

Professor P. L. Hsu, University of North Carolina. 
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9. Some Significance Tests for the Median which are Valid under Very General Condi¬ 
tions (Preliminary Report). 

Mr, John E. Walsh, Princeton University. 

10. Loss of Information in Uiests with Unbalanced Samples (Preliminary Report). 
Mr. John E. Walsh, Princeton University. 

11. Some Theorems on the Bernoullian Multiplicative Process. 

Mr. T. E. Harris, Princeton University. 

Abstracts of all these papers appear elsewhere in this issue of the AnnaU, 
A beer party in honor of the foreign statisticians attending the meeting was 
held in the dining room of Saybrook College on Wednesday evening. A joint 
dinner with the American Mathematical Society and the Mathematical Associ¬ 
ation of America was held on Thursday evening. 

C. C. Craig, 

Acting Secretary. 
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